2023-02-09 17:53:58,874 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-09 17:53:59,106 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb 2023-02-09 17:53:59,121 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 17:53:59,122 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175, deleteOnExit=true 2023-02-09 17:53:59,122 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 17:53:59,122 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/test.cache.data in system properties and HBase conf 2023-02-09 17:53:59,123 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 17:53:59,123 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/hadoop.log.dir in system properties and HBase conf 2023-02-09 17:53:59,124 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 17:53:59,124 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 17:53:59,125 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 17:53:59,237 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 17:53:59,645 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 17:53:59,648 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 17:53:59,649 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 17:53:59,649 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 17:53:59,649 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 17:53:59,649 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 17:53:59,650 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 17:53:59,650 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 17:53:59,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 17:53:59,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 17:53:59,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/nfs.dump.dir in system properties and HBase conf 2023-02-09 17:53:59,651 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/java.io.tmpdir in system properties and HBase conf 2023-02-09 17:53:59,652 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 17:53:59,652 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 17:53:59,652 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 17:54:00,148 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 17:54:00,152 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 17:54:00,426 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 17:54:00,588 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 17:54:00,603 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 17:54:00,640 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 17:54:00,672 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/java.io.tmpdir/Jetty_localhost_35791_hdfs____604nmf/webapp 2023-02-09 17:54:00,833 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35791 2023-02-09 17:54:00,843 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 17:54:00,843 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 17:54:01,228 WARN [Listener at localhost/38249] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 17:54:01,322 WARN [Listener at localhost/38249] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 17:54:01,339 WARN [Listener at localhost/38249] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 17:54:01,347 INFO [Listener at localhost/38249] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 17:54:01,351 INFO [Listener at localhost/38249] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/java.io.tmpdir/Jetty_localhost_36621_datanode____.2wwbxj/webapp 2023-02-09 17:54:01,457 INFO [Listener at localhost/38249] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36621 2023-02-09 17:54:01,776 WARN [Listener at localhost/34199] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 17:54:01,790 WARN [Listener at localhost/34199] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 17:54:01,794 WARN [Listener at localhost/34199] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 17:54:01,796 INFO [Listener at localhost/34199] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 17:54:01,805 INFO [Listener at localhost/34199] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/java.io.tmpdir/Jetty_localhost_39901_datanode____.7r7znp/webapp 2023-02-09 17:54:01,906 INFO [Listener at localhost/34199] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39901 2023-02-09 17:54:01,917 WARN [Listener at localhost/44937] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 17:54:01,930 WARN [Listener at localhost/44937] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 17:54:01,936 WARN [Listener at localhost/44937] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 17:54:01,937 INFO [Listener at localhost/44937] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 17:54:01,942 INFO [Listener at localhost/44937] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/java.io.tmpdir/Jetty_localhost_38415_datanode____.9ok5hi/webapp 2023-02-09 17:54:02,059 INFO [Listener at localhost/44937] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38415 2023-02-09 17:54:02,067 WARN [Listener at localhost/34581] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 17:54:02,292 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x595abaf6248a0fc7: Processing first storage report for DS-f752e557-a81e-487a-94bb-ed02c5a5c838 from datanode 38633d84-bd4c-44ba-82f9-206cacf2085d 2023-02-09 17:54:02,293 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x595abaf6248a0fc7: from storage DS-f752e557-a81e-487a-94bb-ed02c5a5c838 node DatanodeRegistration(127.0.0.1:37849, datanodeUuid=38633d84-bd4c-44ba-82f9-206cacf2085d, infoPort=34353, infoSecurePort=0, ipcPort=34581, storageInfo=lv=-57;cid=testClusterID;nsid=1928245426;c=1675965240225), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 17:54:02,293 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc7a4c01e611623f: Processing first storage report for DS-0597eca3-8035-48fe-8dca-e1468567c5b2 from datanode 1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc7a4c01e611623f: from storage DS-0597eca3-8035-48fe-8dca-e1468567c5b2 node DatanodeRegistration(127.0.0.1:41393, datanodeUuid=1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6, infoPort=33905, infoSecurePort=0, ipcPort=44937, storageInfo=lv=-57;cid=testClusterID;nsid=1928245426;c=1675965240225), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa0b24155e3582912: Processing first storage report for DS-8c368ab6-2529-458e-a26f-971408b07311 from datanode 634cd12e-a559-4a0e-a30b-33f9304f450f 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa0b24155e3582912: from storage DS-8c368ab6-2529-458e-a26f-971408b07311 node DatanodeRegistration(127.0.0.1:34683, datanodeUuid=634cd12e-a559-4a0e-a30b-33f9304f450f, infoPort=34519, infoSecurePort=0, ipcPort=34199, storageInfo=lv=-57;cid=testClusterID;nsid=1928245426;c=1675965240225), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x595abaf6248a0fc7: Processing first storage report for DS-5f7f7357-fead-47ee-9826-52f2b4951ac2 from datanode 38633d84-bd4c-44ba-82f9-206cacf2085d 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x595abaf6248a0fc7: from storage DS-5f7f7357-fead-47ee-9826-52f2b4951ac2 node DatanodeRegistration(127.0.0.1:37849, datanodeUuid=38633d84-bd4c-44ba-82f9-206cacf2085d, infoPort=34353, infoSecurePort=0, ipcPort=34581, storageInfo=lv=-57;cid=testClusterID;nsid=1928245426;c=1675965240225), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc7a4c01e611623f: Processing first storage report for DS-e6da91f6-2dfc-4bc9-8443-437d600fd1b5 from datanode 1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6 2023-02-09 17:54:02,294 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc7a4c01e611623f: from storage DS-e6da91f6-2dfc-4bc9-8443-437d600fd1b5 node DatanodeRegistration(127.0.0.1:41393, datanodeUuid=1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6, infoPort=33905, infoSecurePort=0, ipcPort=44937, storageInfo=lv=-57;cid=testClusterID;nsid=1928245426;c=1675965240225), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 17:54:02,295 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa0b24155e3582912: Processing first storage report for DS-f7d4fd7a-271a-4ff3-a48a-fec9cd6c6ca5 from datanode 634cd12e-a559-4a0e-a30b-33f9304f450f 2023-02-09 17:54:02,295 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa0b24155e3582912: from storage DS-f7d4fd7a-271a-4ff3-a48a-fec9cd6c6ca5 node DatanodeRegistration(127.0.0.1:34683, datanodeUuid=634cd12e-a559-4a0e-a30b-33f9304f450f, infoPort=34519, infoSecurePort=0, ipcPort=34199, storageInfo=lv=-57;cid=testClusterID;nsid=1928245426;c=1675965240225), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 17:54:02,476 DEBUG [Listener at localhost/34581] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb 2023-02-09 17:54:02,545 INFO [Listener at localhost/34581] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/zookeeper_0, clientPort=50368, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 17:54:02,560 INFO [Listener at localhost/34581] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50368 2023-02-09 17:54:02,567 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:02,569 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:03,234 INFO [Listener at localhost/34581] util.FSUtils(479): Created version file at hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07 with version=8 2023-02-09 17:54:03,234 INFO [Listener at localhost/34581] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/hbase-staging 2023-02-09 17:54:03,531 INFO [Listener at localhost/34581] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 17:54:03,982 INFO [Listener at localhost/34581] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 17:54:04,017 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,017 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,018 INFO [Listener at localhost/34581] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 17:54:04,018 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,018 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 17:54:04,160 INFO [Listener at localhost/34581] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 17:54:04,230 DEBUG [Listener at localhost/34581] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 17:54:04,320 INFO [Listener at localhost/34581] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43955 2023-02-09 17:54:04,330 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,332 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,351 INFO [Listener at localhost/34581] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43955 connecting to ZooKeeper ensemble=127.0.0.1:50368 2023-02-09 17:54:04,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:439550x0, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 17:54:04,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43955-0x1003a3806680000 connected 2023-02-09 17:54:04,419 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 17:54:04,420 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:54:04,424 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 17:54:04,432 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43955 2023-02-09 17:54:04,432 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43955 2023-02-09 17:54:04,432 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43955 2023-02-09 17:54:04,433 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43955 2023-02-09 17:54:04,433 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43955 2023-02-09 17:54:04,439 INFO [Listener at localhost/34581] master.HMaster(439): hbase.rootdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07, hbase.cluster.distributed=false 2023-02-09 17:54:04,509 INFO [Listener at localhost/34581] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 17:54:04,509 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,510 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,510 INFO [Listener at localhost/34581] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 17:54:04,510 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,510 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 17:54:04,515 INFO [Listener at localhost/34581] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 17:54:04,517 INFO [Listener at localhost/34581] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44623 2023-02-09 17:54:04,520 INFO [Listener at localhost/34581] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 17:54:04,526 DEBUG [Listener at localhost/34581] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 17:54:04,527 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,529 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,530 INFO [Listener at localhost/34581] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44623 connecting to ZooKeeper ensemble=127.0.0.1:50368 2023-02-09 17:54:04,534 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:446230x0, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 17:54:04,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:44623-0x1003a3806680001 connected 2023-02-09 17:54:04,535 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 17:54:04,537 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:54:04,538 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 17:54:04,538 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44623 2023-02-09 17:54:04,539 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44623 2023-02-09 17:54:04,539 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44623 2023-02-09 17:54:04,540 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44623 2023-02-09 17:54:04,540 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44623 2023-02-09 17:54:04,553 INFO [Listener at localhost/34581] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 17:54:04,554 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,554 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,554 INFO [Listener at localhost/34581] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 17:54:04,555 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,555 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 17:54:04,555 INFO [Listener at localhost/34581] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 17:54:04,557 INFO [Listener at localhost/34581] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35803 2023-02-09 17:54:04,558 INFO [Listener at localhost/34581] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 17:54:04,559 DEBUG [Listener at localhost/34581] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 17:54:04,559 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,562 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,564 INFO [Listener at localhost/34581] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35803 connecting to ZooKeeper ensemble=127.0.0.1:50368 2023-02-09 17:54:04,568 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:358030x0, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 17:54:04,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:35803-0x1003a3806680002 connected 2023-02-09 17:54:04,569 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 17:54:04,570 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:54:04,571 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 17:54:04,572 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35803 2023-02-09 17:54:04,572 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35803 2023-02-09 17:54:04,572 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35803 2023-02-09 17:54:04,573 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35803 2023-02-09 17:54:04,573 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35803 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 17:54:04,586 INFO [Listener at localhost/34581] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 17:54:04,588 INFO [Listener at localhost/34581] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43923 2023-02-09 17:54:04,588 INFO [Listener at localhost/34581] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 17:54:04,589 DEBUG [Listener at localhost/34581] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 17:54:04,590 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,593 INFO [Listener at localhost/34581] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,595 INFO [Listener at localhost/34581] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43923 connecting to ZooKeeper ensemble=127.0.0.1:50368 2023-02-09 17:54:04,598 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:439230x0, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 17:54:04,599 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:439230x0, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 17:54:04,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43923-0x1003a3806680003 connected 2023-02-09 17:54:04,600 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:54:04,601 DEBUG [Listener at localhost/34581] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 17:54:04,602 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43923 2023-02-09 17:54:04,602 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43923 2023-02-09 17:54:04,603 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43923 2023-02-09 17:54:04,604 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43923 2023-02-09 17:54:04,604 DEBUG [Listener at localhost/34581] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43923 2023-02-09 17:54:04,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:04,616 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 17:54:04,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:04,635 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 17:54:04,635 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 17:54:04,635 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 17:54:04,635 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 17:54:04,637 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:04,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 17:54:04,639 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43955,1675965243365 from backup master directory 2023-02-09 17:54:04,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 17:54:04,643 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:04,643 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 17:54:04,643 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 17:54:04,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:04,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 17:54:04,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 17:54:04,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/hbase.id with ID: 1d6d9f6b-8498-4fcf-8193-d3066d73a57a 2023-02-09 17:54:04,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 17:54:04,793 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:04,843 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1573a4e5 to 127.0.0.1:50368 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 17:54:04,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bb4b7cf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 17:54:04,903 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 17:54:04,904 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 17:54:04,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 17:54:04,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 17:54:04,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:54:04,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:54:04,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 17:54:04,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store-tmp 2023-02-09 17:54:05,021 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:05,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 17:54:05,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:54:05,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:54:05,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 17:54:05,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:54:05,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:54:05,023 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675965245021Disabling compacts and flushes for region at 1675965245022 (+1 ms)Disabling writes for close at 1675965245022Writing region close event to WAL at 1675965245022Closed at 1675965245022 2023-02-09 17:54:05,024 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/WALs/jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:05,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43955%2C1675965243365, suffix=, logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/WALs/jenkins-hbase4.apache.org,43955,1675965243365, archiveDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/oldWALs, maxLogs=10 2023-02-09 17:54:05,102 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK] 2023-02-09 17:54:05,102 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK] 2023-02-09 17:54:05,102 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK] 2023-02-09 17:54:05,110 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:54:05,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/WALs/jenkins-hbase4.apache.org,43955,1675965243365/jenkins-hbase4.apache.org%2C43955%2C1675965243365.1675965245056 2023-02-09 17:54:05,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK], DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK], DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK]] 2023-02-09 17:54:05,178 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:05,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:05,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 17:54:05,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 17:54:05,245 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 17:54:05,251 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 17:54:05,274 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 17:54:05,286 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:05,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 17:54:05,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 17:54:05,306 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 17:54:05,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:05,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71378304, jitterRate=0.06361961364746094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 17:54:05,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675965245209Initializing all the Stores at 1675965245212 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965245212Cleaning up temporary data from old regions at 1675965245299 (+87 ms)Cleaning up detritus from prior splits at 1675965245300 (+1 ms)Region opened successfully at 1675965245311 (+11 ms) 2023-02-09 17:54:05,313 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 17:54:05,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:05,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:05,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:05,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 17:54:05,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-02-09 17:54:05,371 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 17:54:05,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 17:54:05,402 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 17:54:05,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 17:54:05,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 17:54:05,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 17:54:05,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 17:54:05,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 17:54:05,443 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:05,444 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 17:54:05,445 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 17:54:05,457 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 17:54:05,461 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 17:54:05,461 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 17:54:05,461 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 17:54:05,462 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:05,461 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 17:54:05,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,43955,1675965243365, sessionid=0x1003a3806680000, setting cluster-up flag (Was=false) 2023-02-09 17:54:05,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:05,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 17:54:05,485 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:05,491 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:05,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 17:54:05,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:05,501 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.hbase-snapshot/.tmp 2023-02-09 17:54:05,508 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(952): ClusterId : 1d6d9f6b-8498-4fcf-8193-d3066d73a57a 2023-02-09 17:54:05,508 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(952): ClusterId : 1d6d9f6b-8498-4fcf-8193-d3066d73a57a 2023-02-09 17:54:05,508 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(952): ClusterId : 1d6d9f6b-8498-4fcf-8193-d3066d73a57a 2023-02-09 17:54:05,513 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 17:54:05,513 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 17:54:05,513 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 17:54:05,519 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 17:54:05,519 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 17:54:05,519 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 17:54:05,520 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 17:54:05,519 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 17:54:05,520 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 17:54:05,524 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 17:54:05,524 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 17:54:05,524 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 17:54:05,526 DEBUG [RS:0;jenkins-hbase4:44623] zookeeper.ReadOnlyZKClient(139): Connect 0x2f64f6e7 to 127.0.0.1:50368 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 17:54:05,526 DEBUG [RS:2;jenkins-hbase4:43923] zookeeper.ReadOnlyZKClient(139): Connect 0x12e7c35f to 127.0.0.1:50368 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 17:54:05,527 DEBUG [RS:1;jenkins-hbase4:35803] zookeeper.ReadOnlyZKClient(139): Connect 0x0a21b651 to 127.0.0.1:50368 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 17:54:05,534 DEBUG [RS:0;jenkins-hbase4:44623] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21a31d7b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 17:54:05,535 DEBUG [RS:2;jenkins-hbase4:43923] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c202f73, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 17:54:05,535 DEBUG [RS:1;jenkins-hbase4:35803] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e65965, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 17:54:05,535 DEBUG [RS:0;jenkins-hbase4:44623] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@259c9d97, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 17:54:05,535 DEBUG [RS:1;jenkins-hbase4:35803] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d9dd21, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 17:54:05,535 DEBUG [RS:2;jenkins-hbase4:43923] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@181fa82f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 17:54:05,558 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:43923 2023-02-09 17:54:05,558 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:35803 2023-02-09 17:54:05,562 INFO [RS:1;jenkins-hbase4:35803] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 17:54:05,563 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:44623 2023-02-09 17:54:05,563 INFO [RS:1;jenkins-hbase4:35803] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 17:54:05,563 INFO [RS:0;jenkins-hbase4:44623] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 17:54:05,562 INFO [RS:2;jenkins-hbase4:43923] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 17:54:05,563 INFO [RS:2;jenkins-hbase4:43923] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 17:54:05,563 INFO [RS:0;jenkins-hbase4:44623] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 17:54:05,563 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 17:54:05,563 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 17:54:05,563 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 17:54:05,566 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43955,1675965243365 with isa=jenkins-hbase4.apache.org/172.31.14.131:43923, startcode=1675965244585 2023-02-09 17:54:05,566 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43955,1675965243365 with isa=jenkins-hbase4.apache.org/172.31.14.131:35803, startcode=1675965244552 2023-02-09 17:54:05,566 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43955,1675965243365 with isa=jenkins-hbase4.apache.org/172.31.14.131:44623, startcode=1675965244508 2023-02-09 17:54:05,586 DEBUG [RS:1;jenkins-hbase4:35803] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 17:54:05,586 DEBUG [RS:2;jenkins-hbase4:43923] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 17:54:05,586 DEBUG [RS:0;jenkins-hbase4:44623] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 17:54:05,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:05,649 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33851, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 17:54:05,649 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33087, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 17:54:05,649 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52601, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 17:54:05,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,652 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965275652 2023-02-09 17:54:05,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 17:54:05,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 17:54:05,665 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:05,666 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 17:54:05,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43955] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 17:54:05,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 17:54:05,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 17:54:05,671 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 17:54:05,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 17:54:05,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 17:54:05,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 17:54:05,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 17:54:05,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 17:54:05,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 17:54:05,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 17:54:05,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 17:54:05,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 17:54:05,689 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675965245689,5,FailOnTimeoutGroup] 2023-02-09 17:54:05,690 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675965245690,5,FailOnTimeoutGroup] 2023-02-09 17:54:05,690 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,690 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 17:54:05,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,706 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 17:54:05,706 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 17:54:05,706 WARN [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 17:54:05,706 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 17:54:05,706 WARN [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 17:54:05,707 WARN [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 17:54:05,730 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:05,731 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:05,731 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07 2023-02-09 17:54:05,755 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:05,758 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 17:54:05,761 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/info 2023-02-09 17:54:05,761 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 17:54:05,762 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:05,763 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 17:54:05,765 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/rep_barrier 2023-02-09 17:54:05,766 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 17:54:05,767 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:05,767 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 17:54:05,769 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/table 2023-02-09 17:54:05,769 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 17:54:05,770 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:05,772 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740 2023-02-09 17:54:05,773 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740 2023-02-09 17:54:05,776 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 17:54:05,778 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 17:54:05,782 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:05,783 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59593019, jitterRate=-0.11199481785297394}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 17:54:05,783 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675965245756Initializing all the Stores at 1675965245757 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675965245757Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965245757Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675965245757Cleaning up temporary data from old regions at 1675965245774 (+17 ms)Cleaning up detritus from prior splits at 1675965245775 (+1 ms)Region opened successfully at 1675965245783 (+8 ms) 2023-02-09 17:54:05,783 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 17:54:05,783 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 17:54:05,784 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 17:54:05,784 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 17:54:05,784 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 17:54:05,785 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 17:54:05,785 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675965245783Disabling compacts and flushes for region at 1675965245783Disabling writes for close at 1675965245784 (+1 ms)Writing region close event to WAL at 1675965245785 (+1 ms)Closed at 1675965245785 2023-02-09 17:54:05,792 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:05,792 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 17:54:05,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 17:54:05,808 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43955,1675965243365 with isa=jenkins-hbase4.apache.org/172.31.14.131:43923, startcode=1675965244585 2023-02-09 17:54:05,808 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43955,1675965243365 with isa=jenkins-hbase4.apache.org/172.31.14.131:35803, startcode=1675965244552 2023-02-09 17:54:05,808 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43955,1675965243365 with isa=jenkins-hbase4.apache.org/172.31.14.131:44623, startcode=1675965244508 2023-02-09 17:54:05,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43955] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,816 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,817 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,825 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07 2023-02-09 17:54:05,825 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07 2023-02-09 17:54:05,825 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38249 2023-02-09 17:54:05,825 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38249 2023-02-09 17:54:05,825 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 17:54:05,826 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 17:54:05,828 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07 2023-02-09 17:54:05,828 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38249 2023-02-09 17:54:05,828 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 17:54:05,829 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 17:54:05,831 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 17:54:05,832 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 17:54:05,837 DEBUG [RS:0;jenkins-hbase4:44623] zookeeper.ZKUtil(162): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,837 WARN [RS:0;jenkins-hbase4:44623] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 17:54:05,837 INFO [RS:0;jenkins-hbase4:44623] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 17:54:05,837 DEBUG [RS:2;jenkins-hbase4:43923] zookeeper.ZKUtil(162): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,837 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,838 DEBUG [RS:1;jenkins-hbase4:35803] zookeeper.ZKUtil(162): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,837 WARN [RS:2;jenkins-hbase4:43923] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 17:54:05,838 WARN [RS:1;jenkins-hbase4:35803] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 17:54:05,838 INFO [RS:2;jenkins-hbase4:43923] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 17:54:05,838 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35803,1675965244552] 2023-02-09 17:54:05,839 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,44623,1675965244508] 2023-02-09 17:54:05,839 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43923,1675965244585] 2023-02-09 17:54:05,838 INFO [RS:1;jenkins-hbase4:35803] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 17:54:05,839 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,839 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,855 DEBUG [RS:1;jenkins-hbase4:35803] zookeeper.ZKUtil(162): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,855 DEBUG [RS:0;jenkins-hbase4:44623] zookeeper.ZKUtil(162): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,855 DEBUG [RS:2;jenkins-hbase4:43923] zookeeper.ZKUtil(162): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,855 DEBUG [RS:1;jenkins-hbase4:35803] zookeeper.ZKUtil(162): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,855 DEBUG [RS:2;jenkins-hbase4:43923] zookeeper.ZKUtil(162): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,855 DEBUG [RS:0;jenkins-hbase4:44623] zookeeper.ZKUtil(162): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,856 DEBUG [RS:1;jenkins-hbase4:35803] zookeeper.ZKUtil(162): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,856 DEBUG [RS:2;jenkins-hbase4:43923] zookeeper.ZKUtil(162): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,856 DEBUG [RS:0;jenkins-hbase4:44623] zookeeper.ZKUtil(162): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,866 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 17:54:05,866 DEBUG [RS:0;jenkins-hbase4:44623] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 17:54:05,866 DEBUG [RS:2;jenkins-hbase4:43923] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 17:54:05,875 INFO [RS:1;jenkins-hbase4:35803] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 17:54:05,875 INFO [RS:0;jenkins-hbase4:44623] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 17:54:05,875 INFO [RS:2;jenkins-hbase4:43923] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 17:54:05,897 INFO [RS:2;jenkins-hbase4:43923] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 17:54:05,897 INFO [RS:0;jenkins-hbase4:44623] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 17:54:05,897 INFO [RS:1;jenkins-hbase4:35803] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 17:54:05,901 INFO [RS:2;jenkins-hbase4:43923] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 17:54:05,901 INFO [RS:0;jenkins-hbase4:44623] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 17:54:05,901 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,901 INFO [RS:1;jenkins-hbase4:35803] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 17:54:05,901 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,902 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,902 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 17:54:05,903 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 17:54:05,902 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 17:54:05,910 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,910 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,910 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,910 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,910 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,911 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 17:54:05,912 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 17:54:05,912 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 17:54:05,912 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,913 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,913 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,913 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,913 DEBUG [RS:0;jenkins-hbase4:44623] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,913 DEBUG [RS:2;jenkins-hbase4:43923] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,912 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,913 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,914 DEBUG [RS:1;jenkins-hbase4:35803] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 17:54:05,914 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,915 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,933 INFO [RS:1;jenkins-hbase4:35803] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 17:54:05,933 INFO [RS:0;jenkins-hbase4:44623] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 17:54:05,933 INFO [RS:2;jenkins-hbase4:43923] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 17:54:05,936 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35803,1675965244552-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,936 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44623,1675965244508-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,936 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43923,1675965244585-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:05,954 INFO [RS:1;jenkins-hbase4:35803] regionserver.Replication(203): jenkins-hbase4.apache.org,35803,1675965244552 started 2023-02-09 17:54:05,954 INFO [RS:0;jenkins-hbase4:44623] regionserver.Replication(203): jenkins-hbase4.apache.org,44623,1675965244508 started 2023-02-09 17:54:05,955 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,35803,1675965244552, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35803, sessionid=0x1003a3806680002 2023-02-09 17:54:05,955 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,44623,1675965244508, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:44623, sessionid=0x1003a3806680001 2023-02-09 17:54:05,955 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 17:54:05,955 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 17:54:05,955 DEBUG [RS:1;jenkins-hbase4:35803] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,955 DEBUG [RS:0;jenkins-hbase4:44623] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,956 DEBUG [RS:1;jenkins-hbase4:35803] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35803,1675965244552' 2023-02-09 17:54:05,956 DEBUG [RS:0;jenkins-hbase4:44623] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44623,1675965244508' 2023-02-09 17:54:05,957 DEBUG [RS:0;jenkins-hbase4:44623] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:05,957 DEBUG [RS:1;jenkins-hbase4:35803] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:05,958 DEBUG [RS:1;jenkins-hbase4:35803] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:05,958 INFO [RS:2;jenkins-hbase4:43923] regionserver.Replication(203): jenkins-hbase4.apache.org,43923,1675965244585 started 2023-02-09 17:54:05,958 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43923,1675965244585, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43923, sessionid=0x1003a3806680003 2023-02-09 17:54:05,958 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 17:54:05,958 DEBUG [RS:2;jenkins-hbase4:43923] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,958 DEBUG [RS:2;jenkins-hbase4:43923] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43923,1675965244585' 2023-02-09 17:54:05,958 DEBUG [RS:2;jenkins-hbase4:43923] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:05,958 DEBUG [RS:0;jenkins-hbase4:44623] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:05,959 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 17:54:05,959 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 17:54:05,959 DEBUG [RS:1;jenkins-hbase4:35803] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:05,959 DEBUG [RS:1;jenkins-hbase4:35803] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35803,1675965244552' 2023-02-09 17:54:05,959 DEBUG [RS:1;jenkins-hbase4:35803] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 17:54:05,959 DEBUG [RS:2;jenkins-hbase4:43923] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:05,959 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 17:54:05,960 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 17:54:05,960 DEBUG [RS:0;jenkins-hbase4:44623] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:05,960 DEBUG [RS:0;jenkins-hbase4:44623] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,44623,1675965244508' 2023-02-09 17:54:05,960 DEBUG [RS:0;jenkins-hbase4:44623] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 17:54:05,961 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 17:54:05,961 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 17:54:05,961 DEBUG [RS:2;jenkins-hbase4:43923] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:05,961 DEBUG [RS:2;jenkins-hbase4:43923] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43923,1675965244585' 2023-02-09 17:54:05,961 DEBUG [RS:2;jenkins-hbase4:43923] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 17:54:05,961 DEBUG [RS:0;jenkins-hbase4:44623] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 17:54:05,961 DEBUG [RS:1;jenkins-hbase4:35803] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 17:54:05,961 DEBUG [RS:2;jenkins-hbase4:43923] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 17:54:05,962 DEBUG [RS:0;jenkins-hbase4:44623] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 17:54:05,962 DEBUG [RS:2;jenkins-hbase4:43923] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 17:54:05,962 INFO [RS:0;jenkins-hbase4:44623] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 17:54:05,962 INFO [RS:2;jenkins-hbase4:43923] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 17:54:05,963 DEBUG [RS:1;jenkins-hbase4:35803] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 17:54:05,963 INFO [RS:0;jenkins-hbase4:44623] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 17:54:05,965 INFO [RS:1;jenkins-hbase4:35803] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 17:54:05,963 INFO [RS:2;jenkins-hbase4:43923] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 17:54:05,965 INFO [RS:1;jenkins-hbase4:35803] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 17:54:05,984 DEBUG [jenkins-hbase4:43955] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-09 17:54:05,989 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:05,996 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:05,996 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:05,996 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:05,996 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:05,999 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35803,1675965244552, state=OPENING 2023-02-09 17:54:06,007 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 17:54:06,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:06,009 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 17:54:06,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:06,076 INFO [RS:2;jenkins-hbase4:43923] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43923%2C1675965244585, suffix=, logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,43923,1675965244585, archiveDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs, maxLogs=32 2023-02-09 17:54:06,076 INFO [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35803%2C1675965244552, suffix=, logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,35803,1675965244552, archiveDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs, maxLogs=32 2023-02-09 17:54:06,076 INFO [RS:0;jenkins-hbase4:44623] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44623%2C1675965244508, suffix=, logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,44623,1675965244508, archiveDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs, maxLogs=32 2023-02-09 17:54:06,108 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK] 2023-02-09 17:54:06,110 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK] 2023-02-09 17:54:06,111 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK] 2023-02-09 17:54:06,111 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK] 2023-02-09 17:54:06,111 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK] 2023-02-09 17:54:06,112 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK] 2023-02-09 17:54:06,112 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK] 2023-02-09 17:54:06,113 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK] 2023-02-09 17:54:06,113 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK] 2023-02-09 17:54:06,128 INFO [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,35803,1675965244552/jenkins-hbase4.apache.org%2C35803%2C1675965244552.1675965246080 2023-02-09 17:54:06,128 INFO [RS:0;jenkins-hbase4:44623] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,44623,1675965244508/jenkins-hbase4.apache.org%2C44623%2C1675965244508.1675965246080 2023-02-09 17:54:06,129 INFO [RS:2;jenkins-hbase4:43923] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,43923,1675965244585/jenkins-hbase4.apache.org%2C43923%2C1675965244585.1675965246080 2023-02-09 17:54:06,129 DEBUG [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK], DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK], DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK]] 2023-02-09 17:54:06,129 DEBUG [RS:0;jenkins-hbase4:44623] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK], DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK], DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK]] 2023-02-09 17:54:06,129 DEBUG [RS:2;jenkins-hbase4:43923] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK], DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK], DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK]] 2023-02-09 17:54:06,196 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:06,199 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 17:54:06,202 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57306, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 17:54:06,214 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 17:54:06,214 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 17:54:06,218 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35803%2C1675965244552.meta, suffix=.meta, logDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,35803,1675965244552, archiveDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs, maxLogs=32 2023-02-09 17:54:06,240 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK] 2023-02-09 17:54:06,243 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK] 2023-02-09 17:54:06,243 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK] 2023-02-09 17:54:06,248 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/WALs/jenkins-hbase4.apache.org,35803,1675965244552/jenkins-hbase4.apache.org%2C35803%2C1675965244552.meta.1675965246221.meta 2023-02-09 17:54:06,249 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34683,DS-8c368ab6-2529-458e-a26f-971408b07311,DISK], DatanodeInfoWithStorage[127.0.0.1:37849,DS-f752e557-a81e-487a-94bb-ed02c5a5c838,DISK], DatanodeInfoWithStorage[127.0.0.1:41393,DS-0597eca3-8035-48fe-8dca-e1468567c5b2,DISK]] 2023-02-09 17:54:06,249 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:06,256 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:06,257 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 17:54:06,280 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 17:54:06,284 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 17:54:06,289 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 17:54:06,289 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:06,289 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 17:54:06,289 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 17:54:06,292 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 17:54:06,294 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/info 2023-02-09 17:54:06,294 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/info 2023-02-09 17:54:06,295 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 17:54:06,296 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:06,296 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 17:54:06,298 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/rep_barrier 2023-02-09 17:54:06,298 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/rep_barrier 2023-02-09 17:54:06,298 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 17:54:06,299 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:06,299 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 17:54:06,301 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/table 2023-02-09 17:54:06,301 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/table 2023-02-09 17:54:06,301 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 17:54:06,302 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:06,305 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740 2023-02-09 17:54:06,308 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740 2023-02-09 17:54:06,312 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 17:54:06,314 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 17:54:06,316 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71437875, jitterRate=0.06450729072093964}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 17:54:06,316 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675965246289Writing region info on filesystem at 1675965246290 (+1 ms)Initializing all the Stores at 1675965246292 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675965246292Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965246292Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675965246292Cleaning up temporary data from old regions at 1675965246310 (+18 ms)Cleaning up detritus from prior splits at 1675965246311 (+1 ms)Running coprocessor post-open hooks at 1675965246316 (+5 ms)Region opened successfully at 1675965246316 2023-02-09 17:54:06,328 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675965246188 2023-02-09 17:54:06,347 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 17:54:06,348 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 17:54:06,348 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,35803,1675965244552, state=OPEN 2023-02-09 17:54:06,351 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 17:54:06,351 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 17:54:06,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 17:54:06,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,35803,1675965244552 in 339 msec 2023-02-09 17:54:06,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 17:54:06,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 556 msec 2023-02-09 17:54:06,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 812 msec 2023-02-09 17:54:06,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675965244632, completionTime=-1 2023-02-09 17:54:06,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-09 17:54:06,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:06,441 DEBUG [hconnection-0x2f5e6fb2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 17:54:06,444 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57314, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 17:54:06,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:06,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965306466 2023-02-09 17:54:06,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965366466 2023-02-09 17:54:06,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 95 msec 2023-02-09 17:54:06,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43955,1675965243365-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:06,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43955,1675965243365-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:06,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43955,1675965243365-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:06,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43955, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:06,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 17:54:06,503 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 17:54:06,513 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 17:54:06,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 17:54:06,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:06,528 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:06,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:06,552 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:06,555 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2 empty. 2023-02-09 17:54:06,555 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:06,556 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 17:54:06,593 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:06,596 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => b77a8e95cd64fdcc1c76452d991420d2, NAME => 'hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:06,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:06,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing b77a8e95cd64fdcc1c76452d991420d2, disabling compactions & flushes 2023-02-09 17:54:06,617 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:06,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:06,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. after waiting 0 ms 2023-02-09 17:54:06,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:06,617 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:06,617 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for b77a8e95cd64fdcc1c76452d991420d2: Waiting for close lock at 1675965246617Disabling compacts and flushes for region at 1675965246617Disabling writes for close at 1675965246617Writing region close event to WAL at 1675965246617Closed at 1675965246617 2023-02-09 17:54:06,622 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:06,636 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675965246624"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965246624"}]},"ts":"1675965246624"} 2023-02-09 17:54:06,665 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:06,668 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:06,672 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965246668"}]},"ts":"1675965246668"} 2023-02-09 17:54:06,677 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 17:54:06,681 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:06,682 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:06,682 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:06,682 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:06,682 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:06,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b77a8e95cd64fdcc1c76452d991420d2, ASSIGN}] 2023-02-09 17:54:06,688 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b77a8e95cd64fdcc1c76452d991420d2, ASSIGN 2023-02-09 17:54:06,689 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b77a8e95cd64fdcc1c76452d991420d2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:54:06,842 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:06,843 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:06,843 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675965246843"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965246843"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965246843"}]},"ts":"1675965246843"} 2023-02-09 17:54:06,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b77a8e95cd64fdcc1c76452d991420d2, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:07,007 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:07,008 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b77a8e95cd64fdcc1c76452d991420d2, NAME => 'hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:07,009 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:07,009 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,009 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:07,010 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,010 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,011 INFO [StoreOpener-b77a8e95cd64fdcc1c76452d991420d2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,013 DEBUG [StoreOpener-b77a8e95cd64fdcc1c76452d991420d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/info 2023-02-09 17:54:07,014 DEBUG [StoreOpener-b77a8e95cd64fdcc1c76452d991420d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/info 2023-02-09 17:54:07,014 INFO [StoreOpener-b77a8e95cd64fdcc1c76452d991420d2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b77a8e95cd64fdcc1c76452d991420d2 columnFamilyName info 2023-02-09 17:54:07,015 INFO [StoreOpener-b77a8e95cd64fdcc1c76452d991420d2-1] regionserver.HStore(310): Store=b77a8e95cd64fdcc1c76452d991420d2/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:07,017 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,017 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:54:07,025 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:07,026 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b77a8e95cd64fdcc1c76452d991420d2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69723533, jitterRate=0.03896160423755646}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 17:54:07,026 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b77a8e95cd64fdcc1c76452d991420d2: Running coprocessor pre-open hook at 1675965247010Writing region info on filesystem at 1675965247010Initializing all the Stores at 1675965247011 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675965247011Cleaning up temporary data from old regions at 1675965247018 (+7 ms)Cleaning up detritus from prior splits at 1675965247019 (+1 ms)Running coprocessor post-open hooks at 1675965247026 (+7 ms)Region opened successfully at 1675965247026 2023-02-09 17:54:07,028 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2., pid=6, masterSystemTime=1675965247001 2023-02-09 17:54:07,032 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:07,032 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:54:07,033 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:07,034 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675965247033"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965247033"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965247033"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965247033"}]},"ts":"1675965247033"} 2023-02-09 17:54:07,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 17:54:07,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b77a8e95cd64fdcc1c76452d991420d2, server=jenkins-hbase4.apache.org,35803,1675965244552 in 189 msec 2023-02-09 17:54:07,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 17:54:07,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b77a8e95cd64fdcc1c76452d991420d2, ASSIGN in 356 msec 2023-02-09 17:54:07,046 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:07,047 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965247046"}]},"ts":"1675965247046"} 2023-02-09 17:54:07,049 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 17:54:07,052 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:07,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 536 msec 2023-02-09 17:54:07,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 17:54:07,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 17:54:07,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:07,169 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:07,188 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 17:54:07,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 36 msec 2023-02-09 17:54:07,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:07,215 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 17:54:07,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-02-09 17:54:07,228 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 17:54:07,231 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 17:54:07,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.587sec 2023-02-09 17:54:07,235 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 17:54:07,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 17:54:07,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 17:54:07,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43955,1675965243365-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 17:54:07,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43955,1675965243365-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 17:54:07,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 17:54:07,315 DEBUG [Listener at localhost/34581] zookeeper.ReadOnlyZKClient(139): Connect 0x4b731b7c to 127.0.0.1:50368 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 17:54:07,320 DEBUG [Listener at localhost/34581] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@328d140f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 17:54:07,334 DEBUG [hconnection-0x6d6b0340-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 17:54:07,345 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57330, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 17:54:07,354 INFO [Listener at localhost/34581] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:54:07,383 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=257, ProcessCount=168, AvailableMemoryMB=3610 2023-02-09 17:54:07,388 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:07,389 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:07,389 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:07,398 DEBUG [Listener at localhost/34581] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 17:54:07,402 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48236, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 17:54:07,417 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:54:07,418 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:54:07,419 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:54:07,424 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:07,429 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:07,432 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,433 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d empty. 2023-02-09 17:54:07,435 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,435 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 17:54:07,458 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:07,460 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => d08cc38916d551c13ac0de49fa30546d, NAME => 'testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:07,480 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:07,480 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing d08cc38916d551c13ac0de49fa30546d, disabling compactions & flushes 2023-02-09 17:54:07,480 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,480 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,480 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. after waiting 0 ms 2023-02-09 17:54:07,481 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,481 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,481 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for d08cc38916d551c13ac0de49fa30546d: Waiting for close lock at 1675965247480Disabling compacts and flushes for region at 1675965247480Disabling writes for close at 1675965247481 (+1 ms)Writing region close event to WAL at 1675965247481Closed at 1675965247481 2023-02-09 17:54:07,485 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:07,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675965247487"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965247487"}]},"ts":"1675965247487"} 2023-02-09 17:54:07,489 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:07,491 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:07,491 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965247491"}]},"ts":"1675965247491"} 2023-02-09 17:54:07,493 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-09 17:54:07,498 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:07,499 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:07,500 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:07,500 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:07,500 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:07,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, ASSIGN}] 2023-02-09 17:54:07,503 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, ASSIGN 2023-02-09 17:54:07,504 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:07,654 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:07,656 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d08cc38916d551c13ac0de49fa30546d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:07,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675965247655"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965247655"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965247655"}]},"ts":"1675965247655"} 2023-02-09 17:54:07,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure d08cc38916d551c13ac0de49fa30546d, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:07,813 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:07,814 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 17:54:07,817 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48704, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 17:54:07,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d08cc38916d551c13ac0de49fa30546d, NAME => 'testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:07,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:07,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:07,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,826 INFO [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,829 DEBUG [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1 2023-02-09 17:54:07,829 DEBUG [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1 2023-02-09 17:54:07,829 INFO [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d08cc38916d551c13ac0de49fa30546d columnFamilyName cf1 2023-02-09 17:54:07,830 INFO [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] regionserver.HStore(310): Store=d08cc38916d551c13ac0de49fa30546d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:07,830 INFO [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,832 DEBUG [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2 2023-02-09 17:54:07,832 DEBUG [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2 2023-02-09 17:54:07,832 INFO [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d08cc38916d551c13ac0de49fa30546d columnFamilyName cf2 2023-02-09 17:54:07,833 INFO [StoreOpener-d08cc38916d551c13ac0de49fa30546d-1] regionserver.HStore(310): Store=d08cc38916d551c13ac0de49fa30546d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:07,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:07,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:07,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:07,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d08cc38916d551c13ac0de49fa30546d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69946305, jitterRate=0.04228116571903229}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:07,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d08cc38916d551c13ac0de49fa30546d: Running coprocessor pre-open hook at 1675965247825Writing region info on filesystem at 1675965247825Initializing all the Stores at 1675965247826 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965247826Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965247826Cleaning up temporary data from old regions at 1675965247837 (+11 ms)Cleaning up detritus from prior splits at 1675965247838 (+1 ms)Running coprocessor post-open hooks at 1675965247845 (+7 ms)Region opened successfully at 1675965247845 2023-02-09 17:54:07,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d., pid=11, masterSystemTime=1675965247813 2023-02-09 17:54:07,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,852 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d08cc38916d551c13ac0de49fa30546d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:07,853 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675965247852"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965247852"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965247852"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965247852"}]},"ts":"1675965247852"} 2023-02-09 17:54:07,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 17:54:07,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure d08cc38916d551c13ac0de49fa30546d, server=jenkins-hbase4.apache.org,44623,1675965244508 in 196 msec 2023-02-09 17:54:07,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 17:54:07,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, ASSIGN in 359 msec 2023-02-09 17:54:07,863 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:07,863 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965247863"}]},"ts":"1675965247863"} 2023-02-09 17:54:07,866 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-09 17:54:07,869 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:07,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 447 msec 2023-02-09 17:54:07,929 DEBUG [Listener at localhost/34581] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 17:54:07,934 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48710, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 17:54:07,952 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:07,952 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:07,955 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:07,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,964 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:07,964 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:07,965 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 17:54:07,965 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 17:54:07,966 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,966 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,968 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:07,968 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:07,968 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:07,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:07,969 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:07,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:07,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:07,969 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:07,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,970 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:07,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,973 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:07,974 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:07,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:07,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:07,974 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:07,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:07,975 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:07,975 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:07,974 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:07,975 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:07,975 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:07,975 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:07,976 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:07,976 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:07,976 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:07,976 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:07,976 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. started... 2023-02-09 17:54:07,978 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:54:07,978 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,979 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,979 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,979 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:07,980 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:07,980 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,053 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/e80a0faa1b0b4485a4794021f605fd58 2023-02-09 17:54:08,147 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/c6656382ab014377a27090706eeb4ba4 2023-02-09 17:54:08,171 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/e80a0faa1b0b4485a4794021f605fd58 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58 2023-02-09 17:54:08,182 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:08,185 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/c6656382ab014377a27090706eeb4ba4 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4 2023-02-09 17:54:08,194 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:08,196 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 218ms, sequenceid=6, compaction requested=false 2023-02-09 17:54:08,197 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,199 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d08cc38916d551c13ac0de49fa30546d: 2023-02-09 17:54:08,199 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,203 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 17:54:08,206 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@302fc403[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:08,207 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,213 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:08,213 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,213 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:08,217 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,217 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,217 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,217 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,218 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,218 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,218 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,218 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,218 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,219 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:08,224 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 17:54:08,224 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:08,224 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:08,224 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,224 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:08,226 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,226 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,226 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,226 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,226 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,226 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,227 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,227 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,228 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,229 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,232 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,233 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,233 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,245 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,245 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,245 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,245 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,246 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,246 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,246 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,245 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,246 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,246 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,247 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:08,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,246 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,247 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,247 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,247 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,247 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,248 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,248 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,248 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,248 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,249 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:08,249 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:08,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,349 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:08,350 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:08,350 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:08,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:08,379 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:08,380 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:08,381 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:08,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:08,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,384 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,384 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 17:54:08,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 17:54:08,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,386 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,387 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,387 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,387 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,387 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,388 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,388 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:08,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,390 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,390 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,390 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,390 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,391 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,391 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,391 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,390 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:08,392 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,392 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. started... 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:08,392 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,392 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,393 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,393 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,393 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,396 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,415 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/80d70edb70c84e1a8d61154e79b81cd7 2023-02-09 17:54:08,445 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/fba301782a3a4b96a88a607dd8b39643 2023-02-09 17:54:08,454 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/80d70edb70c84e1a8d61154e79b81cd7 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7 2023-02-09 17:54:08,462 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:08,463 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/fba301782a3a4b96a88a607dd8b39643 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643 2023-02-09 17:54:08,472 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:08,473 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 81ms, sequenceid=19, compaction requested=false 2023-02-09 17:54:08,474 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d08cc38916d551c13ac0de49fa30546d: 2023-02-09 17:54:08,474 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,474 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,478 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 17:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@426a57f3[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:08,482 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:08,482 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,484 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,484 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:08,486 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,486 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,486 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,486 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,486 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,486 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,487 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,488 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,488 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,488 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:08,493 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 17:54:08,493 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:08,493 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:08,493 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,494 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,496 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,497 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,498 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,499 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,501 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,502 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,502 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,512 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,512 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:08,512 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,513 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,512 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:08,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,614 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:08,614 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:08,614 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:08,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:08,638 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:08,638 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:08,639 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:08,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:08,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,642 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,642 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,643 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 17:54:08,643 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 17:54:08,643 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,643 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,646 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,646 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,646 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,646 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,647 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,647 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:08,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,650 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,650 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,650 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,651 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:08,651 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. started... 2023-02-09 17:54:08,650 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,651 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,654 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,654 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,654 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,654 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,655 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,655 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,670 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/801a9c67b5c2440b981e34eae88f6aa5 2023-02-09 17:54:08,694 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/addbdbbc7743470590b0a13701163363 2023-02-09 17:54:08,703 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/801a9c67b5c2440b981e34eae88f6aa5 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5 2023-02-09 17:54:08,710 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:08,711 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/addbdbbc7743470590b0a13701163363 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363 2023-02-09 17:54:08,720 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:08,722 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 71ms, sequenceid=32, compaction requested=true 2023-02-09 17:54:08,722 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d08cc38916d551c13ac0de49fa30546d: 2023-02-09 17:54:08,722 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,722 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:08,723 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:08,723 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,723 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,723 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,725 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,725 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,726 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 17:54:08,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e5e6450[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:08,728 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:08,728 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,731 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,731 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:08,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:08,731 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,735 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,735 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:08,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:08,741 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 17:54:08,742 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:08,742 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:08,742 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,742 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:08,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,744 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,745 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,746 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,749 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,749 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,750 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,756 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,757 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,757 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,756 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,757 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,757 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,757 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,757 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,757 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,757 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,758 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:08,758 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,758 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,758 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,758 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,758 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,758 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,759 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:08,759 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:08,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,859 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:08,859 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:08,859 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:08,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:08,882 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:08,882 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:08,883 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:08,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:08,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,886 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:08,886 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,887 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 17:54:08,887 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 17:54:08,888 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,888 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,889 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,889 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,889 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,889 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:08,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:08,889 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,890 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,892 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,892 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,892 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:08,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:08,893 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,893 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. started... 2023-02-09 17:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:08,894 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:08,895 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,895 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,897 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,897 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,898 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,898 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,898 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,898 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,914 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/79da2298a331464b8d3deeb34a23c8b8 2023-02-09 17:54:08,944 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 2023-02-09 17:54:08,953 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/79da2298a331464b8d3deeb34a23c8b8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8 2023-02-09 17:54:08,960 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:08,962 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 2023-02-09 17:54:08,971 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:08,972 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 79ms, sequenceid=45, compaction requested=true 2023-02-09 17:54:08,973 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d08cc38916d551c13ac0de49fa30546d: 2023-02-09 17:54:08,973 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:08,973 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:08,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:08,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:08,974 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:08,979 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,979 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,979 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,979 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 17:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67c87dd7[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:08,982 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:08,982 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,983 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,983 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,983 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:08,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,984 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,984 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:08,984 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,987 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,988 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,988 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:08,988 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:08,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,988 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:08,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:08,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:08,994 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 17:54:08,994 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:08,994 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:08,994 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,994 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,997 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,999 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:08,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:08,999 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,000 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,003 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,003 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,004 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,007 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,008 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,009 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,009 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,009 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,009 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,010 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,010 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,010 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,010 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,010 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,010 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:09,010 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:09,010 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,011 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,012 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:09,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,012 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:09,112 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:09,112 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:09,112 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:09,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:09,135 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:09,136 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:09,137 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:09,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:09,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,139 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:09,139 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,140 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 17:54:09,140 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 17:54:09,141 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,141 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,142 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,142 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,142 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,142 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,143 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,143 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:09,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:09,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,144 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:09,146 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:09,146 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,147 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,147 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:09,147 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,146 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,148 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:09,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:09,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:09,147 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:09,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:09,150 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:09,150 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:09,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:09,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:09,150 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,150 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. started... 2023-02-09 17:54:09,150 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:09,151 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,152 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,152 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,152 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:09,152 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,153 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:09,170 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb 2023-02-09 17:54:09,201 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/31235553b25746508cbd40994e4f5327 2023-02-09 17:54:09,210 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb 2023-02-09 17:54:09,219 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:09,220 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/31235553b25746508cbd40994e4f5327 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327 2023-02-09 17:54:09,228 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:09,229 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 79ms, sequenceid=58, compaction requested=true 2023-02-09 17:54:09,229 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d08cc38916d551c13ac0de49fa30546d: 2023-02-09 17:54:09,229 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,229 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:09,230 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:09,230 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,230 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:09,230 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:09,233 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,233 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:09,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:09,234 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,234 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:09,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:09,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 17:54:09,237 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2989b2f1[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:09,237 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:09,237 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:09,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:09,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:09,238 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,239 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:09,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:09,239 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:09,242 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:09,242 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:09,242 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,242 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,242 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:09,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:09,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:09,242 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:09,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:09,249 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 17:54:09,249 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:09,249 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:09,249 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,249 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:09,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:09,253 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,253 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,254 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,256 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,258 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,258 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,259 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,265 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:09,265 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,265 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,266 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,266 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:09,266 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,266 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,266 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:09,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,266 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,267 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,267 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,267 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,367 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:09,367 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:09,367 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:09,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:09,390 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:09,390 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:09,391 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:09,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:09,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,394 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:09,394 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,395 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 17:54:09,395 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 17:54:09,395 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,395 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,397 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,397 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,397 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,397 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,397 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,397 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,399 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,399 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:09,399 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,399 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,399 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,399 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:09,399 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,399 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:09,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 17:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 17:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 17:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:09,402 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:09,402 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,402 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. started... 2023-02-09 17:54:09,402 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:09,403 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,405 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:09,424 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/554c50f6a65d41bebedfa45b633d3923 2023-02-09 17:54:09,446 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/efd00090578b4832b1e213915cf032ec 2023-02-09 17:54:09,455 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/554c50f6a65d41bebedfa45b633d3923 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923 2023-02-09 17:54:09,463 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:09,464 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/efd00090578b4832b1e213915cf032ec as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec 2023-02-09 17:54:09,472 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:09,473 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 71ms, sequenceid=71, compaction requested=true 2023-02-09 17:54:09,473 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d08cc38916d551c13ac0de49fa30546d: 2023-02-09 17:54:09,474 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,474 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:09,475 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:09,475 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,475 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 17:54:09,475 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 17:54:09,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,476 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:09,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:09,477 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:09,477 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:09,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 17:54:09,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@394cbdfc[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:09,479 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:09,479 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:09,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,481 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:09,481 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:09,485 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:09,485 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:09,485 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,485 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,485 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:09,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:09,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:09,485 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,486 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 17:54:09,486 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,486 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:09,486 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 17:54:09,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:09,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:09,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:09,491 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 17:54:09,491 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:09,491 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:09,491 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,491 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:09,494 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,493 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,494 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,495 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:09,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,497 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:09,498 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,498 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,501 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,501 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,501 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,502 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,502 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,502 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:09,501 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:09,502 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:09,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:09,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,502 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 17:54:09,503 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:09,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:09,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:09,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:09,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:09,504 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 17:54:09,604 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:09,605 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:09,605 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:09,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:09,640 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,649 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:09,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:09,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:09,651 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:09,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, UNASSIGN}] 2023-02-09 17:54:09,661 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, UNASSIGN 2023-02-09 17:54:09,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d08cc38916d551c13ac0de49fa30546d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:09,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675965249663"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965249663"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965249663"}]},"ts":"1675965249663"} 2023-02-09 17:54:09,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure d08cc38916d551c13ac0de49fa30546d, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:09,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d08cc38916d551c13ac0de49fa30546d, disabling compactions & flushes 2023-02-09 17:54:09,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. after waiting 0 ms 2023-02-09 17:54:09,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing d08cc38916d551c13ac0de49fa30546d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:54:09,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/945bcbc2d2524784ab5e366f1561f6dd 2023-02-09 17:54:09,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/56487b1a99354c748abb2b0622aded23 2023-02-09 17:54:09,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf1/945bcbc2d2524784ab5e366f1561f6dd as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd 2023-02-09 17:54:09,884 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:54:09,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.tmp/cf2/56487b1a99354c748abb2b0622aded23 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23 2023-02-09 17:54:09,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:54:09,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 69ms, sequenceid=82, compaction requested=true 2023-02-09 17:54:09,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:54:09,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:09,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. 2023-02-09 17:54:09,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d08cc38916d551c13ac0de49fa30546d: Waiting for close lock at 1675965249828Running coprocessor pre-close hooks at 1675965249828Disabling compacts and flushes for region at 1675965249828Disabling writes for close at 1675965249828Obtaining lock to block concurrent updates at 1675965249828Preparing flush snapshotting stores in d08cc38916d551c13ac0de49fa30546d at 1675965249828Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675965249828Flushing stores of testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. at 1675965249829 (+1 ms)Flushing d08cc38916d551c13ac0de49fa30546d/cf1: creating writer at 1675965249830 (+1 ms)Flushing d08cc38916d551c13ac0de49fa30546d/cf1: appending metadata at 1675965249833 (+3 ms)Flushing d08cc38916d551c13ac0de49fa30546d/cf1: closing flushed file at 1675965249833Flushing d08cc38916d551c13ac0de49fa30546d/cf2: creating writer at 1675965249856 (+23 ms)Flushing d08cc38916d551c13ac0de49fa30546d/cf2: appending metadata at 1675965249858 (+2 ms)Flushing d08cc38916d551c13ac0de49fa30546d/cf2: closing flushed file at 1675965249858Flushing d08cc38916d551c13ac0de49fa30546d/cf1: reopening flushed file at 1675965249878 (+20 ms)Flushing d08cc38916d551c13ac0de49fa30546d/cf2: reopening flushed file at 1675965249889 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d08cc38916d551c13ac0de49fa30546d in 69ms, sequenceid=82, compaction requested=true at 1675965249897 (+8 ms)Writing region close event to WAL at 1675965249914 (+17 ms)Running coprocessor post-close hooks at 1675965249920 (+6 ms)Closed at 1675965249921 (+1 ms) 2023-02-09 17:54:09,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d08cc38916d551c13ac0de49fa30546d, regionState=CLOSED 2023-02-09 17:54:09,928 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675965249926"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965249926"}]},"ts":"1675965249926"} 2023-02-09 17:54:09,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 17:54:09,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure d08cc38916d551c13ac0de49fa30546d, server=jenkins-hbase4.apache.org,44623,1675965244508 in 264 msec 2023-02-09 17:54:09,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-09 17:54:09,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d08cc38916d551c13ac0de49fa30546d, UNASSIGN in 274 msec 2023-02-09 17:54:09,959 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=d08cc38916d551c13ac0de49fa30546d, threads=14 2023-02-09 17:54:09,960 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,960 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,960 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,961 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,962 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,962 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,962 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,962 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,963 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,963 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,967 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,967 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,968 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:09,972 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,000 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923, top=true 2023-02-09 17:54:10,006 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327, top=true 2023-02-09 17:54:10,019 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec, top=true 2023-02-09 17:54:10,029 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd, top=true 2023-02-09 17:54:10,037 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb, top=true 2023-02-09 17:54:10,074 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23, top=true 2023-02-09 17:54:10,200 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.splits/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923 for child: 4b2902b446be90fa1b555fe9a2d595e6, parent: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,200 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,212 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.splits/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb for child: 4b2902b446be90fa1b555fe9a2d595e6, parent: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,218 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.splits/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327 for child: 4b2902b446be90fa1b555fe9a2d595e6, parent: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,218 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,218 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,216 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.splits/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec for child: 4b2902b446be90fa1b555fe9a2d595e6, parent: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,223 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,248 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.splits/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd for child: 4b2902b446be90fa1b555fe9a2d595e6, parent: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,248 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,262 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/.splits/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23 for child: 4b2902b446be90fa1b555fe9a2d595e6, parent: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,262 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,328 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,344 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,345 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,353 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,366 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,381 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,389 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,687 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8 for region: d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:10,687 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region d08cc38916d551c13ac0de49fa30546d Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:54:10,737 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:54:10,739 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:54:10,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675965250741"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675965250741"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675965250741"}]},"ts":"1675965250741"} 2023-02-09 17:54:10,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965250741"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965250741"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965250741"}]},"ts":"1675965250741"} 2023-02-09 17:54:10,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965250741"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965250741"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965250741"}]},"ts":"1675965250741"} 2023-02-09 17:54:10,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, ASSIGN}] 2023-02-09 17:54:10,796 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, ASSIGN 2023-02-09 17:54:10,796 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:10,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, ASSIGN 2023-02-09 17:54:10,798 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:10,947 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:54:10,948 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=4b2902b446be90fa1b555fe9a2d595e6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:10,949 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965250948"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965250948"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965250948"}]},"ts":"1675965250948"} 2023-02-09 17:54:10,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 4b2902b446be90fa1b555fe9a2d595e6, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:10,955 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1d292a7367afe6611400f38936e4376b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:10,955 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965250955"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965250955"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965250955"}]},"ts":"1675965250955"} 2023-02-09 17:54:10,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 1d292a7367afe6611400f38936e4376b, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:11,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4b2902b446be90fa1b555fe9a2d595e6, NAME => 'testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.', STARTKEY => '26', ENDKEY => ''} 2023-02-09 17:54:11,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:11,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:11,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,111 INFO [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,113 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1 2023-02-09 17:54:11,113 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1 2023-02-09 17:54:11,113 INFO [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b2902b446be90fa1b555fe9a2d595e6 columnFamilyName cf1 2023-02-09 17:54:11,153 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8-top 2023-02-09 17:54:11,162 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5-top 2023-02-09 17:54:11,169 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7-top 2023-02-09 17:54:11,176 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58-top 2023-02-09 17:54:11,183 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923 2023-02-09 17:54:11,189 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd 2023-02-09 17:54:11,194 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb 2023-02-09 17:54:11,195 INFO [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(310): Store=4b2902b446be90fa1b555fe9a2d595e6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:11,195 INFO [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,196 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2 2023-02-09 17:54:11,196 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2 2023-02-09 17:54:11,196 INFO [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b2902b446be90fa1b555fe9a2d595e6 columnFamilyName cf2 2023-02-09 17:54:11,210 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586-top 2023-02-09 17:54:11,216 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363-top 2023-02-09 17:54:11,222 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4-top 2023-02-09 17:54:11,227 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643-top 2023-02-09 17:54:11,232 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327 2023-02-09 17:54:11,237 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23 2023-02-09 17:54:11,242 DEBUG [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec 2023-02-09 17:54:11,243 INFO [StoreOpener-4b2902b446be90fa1b555fe9a2d595e6-1] regionserver.HStore(310): Store=4b2902b446be90fa1b555fe9a2d595e6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:11,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:11,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:11,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4b2902b446be90fa1b555fe9a2d595e6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60246452, jitterRate=-0.10225790739059448}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:11,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4b2902b446be90fa1b555fe9a2d595e6: Running coprocessor pre-open hook at 1675965251110Writing region info on filesystem at 1675965251110Initializing all the Stores at 1675965251111 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965251111Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965251111Cleaning up temporary data from old regions at 1675965251247 (+136 ms)Cleaning up detritus from prior splits at 1675965251247Running coprocessor post-open hooks at 1675965251254 (+7 ms)Region opened successfully at 1675965251254 2023-02-09 17:54:11,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6., pid=17, masterSystemTime=1675965251103 2023-02-09 17:54:11,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,259 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:11,259 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:11,263 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,263 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,263 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1912): 4b2902b446be90fa1b555fe9a2d595e6/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:11,263 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1912): 4b2902b446be90fa1b555fe9a2d595e6/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:11,263 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4b2902b446be90fa1b555fe9a2d595e6/cf1 in testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,263 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4b2902b446be90fa1b555fe9a2d595e6/cf2 in testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,264 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/.tmp, totalSize=34.9 K 2023-02-09 17:54:11,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,264 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/.tmp, totalSize=34.9 K 2023-02-09 17:54:11,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:11,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1d292a7367afe6611400f38936e4376b, NAME => 'testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.', STARTKEY => '', ENDKEY => '26'} 2023-02-09 17:54:11,266 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=4b2902b446be90fa1b555fe9a2d595e6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:11,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:11,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,268 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965251266"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965251266"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965251266"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965251266"}]},"ts":"1675965251266"} 2023-02-09 17:54:11,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:11,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,271 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965247934 2023-02-09 17:54:11,271 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965247934 2023-02-09 17:54:11,273 INFO [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,274 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965248358 2023-02-09 17:54:11,274 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965248358 2023-02-09 17:54:11,275 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1 2023-02-09 17:54:11,275 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1 2023-02-09 17:54:11,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-09 17:54:11,275 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965248617 2023-02-09 17:54:11,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 4b2902b446be90fa1b555fe9a2d595e6, server=jenkins-hbase4.apache.org,44623,1675965244508 in 320 msec 2023-02-09 17:54:11,275 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965248617 2023-02-09 17:54:11,276 INFO [StoreOpener-1d292a7367afe6611400f38936e4376b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d292a7367afe6611400f38936e4376b columnFamilyName cf1 2023-02-09 17:54:11,276 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965248862 2023-02-09 17:54:11,276 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965248862 2023-02-09 17:54:11,278 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965249115 2023-02-09 17:54:11,278 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965249115 2023-02-09 17:54:11,279 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965249370 2023-02-09 17:54:11,279 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965249370 2023-02-09 17:54:11,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, ASSIGN in 481 msec 2023-02-09 17:54:11,280 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965249608 2023-02-09 17:54:11,281 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965249608 2023-02-09 17:54:11,333 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8-bottom 2023-02-09 17:54:11,348 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5-bottom 2023-02-09 17:54:11,348 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] throttle.PressureAwareThroughputController(145): 4b2902b446be90fa1b555fe9a2d595e6#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:11,351 DEBUG [RS:0;jenkins-hbase4:44623] compactions.CompactionProgress(74): totalCompactingKVs=44 less than currentCompactedKVs=45 2023-02-09 17:54:11,361 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7-bottom 2023-02-09 17:54:11,364 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4b2902b446be90fa1b555fe9a2d595e6#cf1#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:11,369 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58-bottom 2023-02-09 17:54:11,373 INFO [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(310): Store=1d292a7367afe6611400f38936e4376b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:11,373 INFO [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,375 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2 2023-02-09 17:54:11,375 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2 2023-02-09 17:54:11,375 INFO [StoreOpener-1d292a7367afe6611400f38936e4376b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d292a7367afe6611400f38936e4376b columnFamilyName cf2 2023-02-09 17:54:11,382 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/.tmp/cf2/1594ecdceae64a35a3c4fd22db76084d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1594ecdceae64a35a3c4fd22db76084d 2023-02-09 17:54:11,386 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/.tmp/cf1/98342e15793747ef80b4d093df133e3b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/98342e15793747ef80b4d093df133e3b 2023-02-09 17:54:11,395 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586-bottom 2023-02-09 17:54:11,400 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363-bottom 2023-02-09 17:54:11,404 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4b2902b446be90fa1b555fe9a2d595e6/cf2 of 4b2902b446be90fa1b555fe9a2d595e6 into 1594ecdceae64a35a3c4fd22db76084d(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:11,404 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4b2902b446be90fa1b555fe9a2d595e6/cf1 of 4b2902b446be90fa1b555fe9a2d595e6 into 98342e15793747ef80b4d093df133e3b(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:11,406 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4b2902b446be90fa1b555fe9a2d595e6: 2023-02-09 17:54:11,406 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4b2902b446be90fa1b555fe9a2d595e6: 2023-02-09 17:54:11,406 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6., storeName=4b2902b446be90fa1b555fe9a2d595e6/cf2, priority=9, startTime=1675965251258; duration=0sec 2023-02-09 17:54:11,406 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6., storeName=4b2902b446be90fa1b555fe9a2d595e6/cf1, priority=9, startTime=1675965251258; duration=0sec 2023-02-09 17:54:11,407 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,407 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,408 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4-bottom 2023-02-09 17:54:11,412 DEBUG [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643-bottom 2023-02-09 17:54:11,413 INFO [StoreOpener-1d292a7367afe6611400f38936e4376b-1] regionserver.HStore(310): Store=1d292a7367afe6611400f38936e4376b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:11,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:11,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:11,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1d292a7367afe6611400f38936e4376b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67653586, jitterRate=0.008116990327835083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:11,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1d292a7367afe6611400f38936e4376b: Running coprocessor pre-open hook at 1675965251268Writing region info on filesystem at 1675965251268Initializing all the Stores at 1675965251272 (+4 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965251272Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965251273 (+1 ms)Cleaning up temporary data from old regions at 1675965251416 (+143 ms)Cleaning up detritus from prior splits at 1675965251416Running coprocessor post-open hooks at 1675965251420 (+4 ms)Region opened successfully at 1675965251420 2023-02-09 17:54:11,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b., pid=18, masterSystemTime=1675965251103 2023-02-09 17:54:11,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,424 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 17:54:11,424 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 17:54:11,424 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,424 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,424 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1912): 1d292a7367afe6611400f38936e4376b/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:11,424 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1d292a7367afe6611400f38936e4376b/cf1 in testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,425 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/.tmp, totalSize=19.9 K 2023-02-09 17:54:11,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:11,424 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1912): 1d292a7367afe6611400f38936e4376b/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:11,426 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1d292a7367afe6611400f38936e4376b/cf2 in testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,426 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965247934 2023-02-09 17:54:11,426 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/.tmp, totalSize=19.9 K 2023-02-09 17:54:11,427 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965248358 2023-02-09 17:54:11,428 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965247934 2023-02-09 17:54:11,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:11,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1d292a7367afe6611400f38936e4376b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:11,430 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965251429"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965251429"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965251429"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965251429"}]},"ts":"1675965251429"} 2023-02-09 17:54:11,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-09 17:54:11,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 1d292a7367afe6611400f38936e4376b, server=jenkins-hbase4.apache.org,44623,1675965244508 in 475 msec 2023-02-09 17:54:11,440 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965248617 2023-02-09 17:54:11,440 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965248358 2023-02-09 17:54:11,441 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965248862 2023-02-09 17:54:11,442 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965248617 2023-02-09 17:54:11,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-09 17:54:11,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, ASSIGN in 643 msec 2023-02-09 17:54:11,442 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965248862 2023-02-09 17:54:11,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 in 1.8020 sec 2023-02-09 17:54:11,457 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1d292a7367afe6611400f38936e4376b#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:11,461 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] throttle.PressureAwareThroughputController(145): 1d292a7367afe6611400f38936e4376b#cf2#compaction#17 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:11,478 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/.tmp/cf2/db928ebf5c6c477db915cbe0d7a4cc10 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/db928ebf5c6c477db915cbe0d7a4cc10 2023-02-09 17:54:11,481 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/.tmp/cf1/7b597d6a003a464c9bb2cf6185a26ee8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/7b597d6a003a464c9bb2cf6185a26ee8 2023-02-09 17:54:11,485 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1d292a7367afe6611400f38936e4376b/cf2 of 1d292a7367afe6611400f38936e4376b into db928ebf5c6c477db915cbe0d7a4cc10(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:11,485 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1d292a7367afe6611400f38936e4376b: 2023-02-09 17:54:11,485 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b., storeName=1d292a7367afe6611400f38936e4376b/cf2, priority=12, startTime=1675965251424; duration=0sec 2023-02-09 17:54:11,485 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,488 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1d292a7367afe6611400f38936e4376b/cf1 of 1d292a7367afe6611400f38936e4376b into 7b597d6a003a464c9bb2cf6185a26ee8(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:11,488 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1d292a7367afe6611400f38936e4376b: 2023-02-09 17:54:11,488 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b., storeName=1d292a7367afe6611400f38936e4376b/cf1, priority=12, startTime=1675965251423; duration=0sec 2023-02-09 17:54:11,488 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:11,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44623] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:48710 deadline: 1675965311651, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d. is not online on jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:11,769 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 17:54:11,781 INFO [Listener at localhost/34581] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 17:54:11,784 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 17:54:11,810 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:11,810 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:11,811 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:11,819 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:11,819 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:11,820 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-09 17:54:11,824 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-09 17:54:11,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:11,836 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965251836"}]},"ts":"1675965251836"} 2023-02-09 17:54:11,838 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-09 17:54:11,840 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:11,840 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:11,842 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:54:11,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 17:54:11,844 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-09 17:54:11,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, UNASSIGN}] 2023-02-09 17:54:11,849 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, UNASSIGN 2023-02-09 17:54:11,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=4b2902b446be90fa1b555fe9a2d595e6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:11,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965251849"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965251849"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965251849"}]},"ts":"1675965251849"} 2023-02-09 17:54:11,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 4b2902b446be90fa1b555fe9a2d595e6, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:11,856 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, UNASSIGN 2023-02-09 17:54:11,857 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=1d292a7367afe6611400f38936e4376b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:11,857 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965251857"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965251857"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965251857"}]},"ts":"1675965251857"} 2023-02-09 17:54:11,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 1d292a7367afe6611400f38936e4376b, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:11,908 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 17:54:11,909 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 17:54:11,946 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:11,947 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:11,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 17:54:12,005 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:12,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4b2902b446be90fa1b555fe9a2d595e6, disabling compactions & flushes 2023-02-09 17:54:12,005 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:12,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:12,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. after waiting 0 ms 2023-02-09 17:54:12,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:12,017 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd] to archive 2023-02-09 17:54:12,018 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:12,024 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,026 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,028 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,030 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,032 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-c1b70d73bde54e0dad02ad3eb42b1bfb 2023-02-09 17:54:12,034 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-554c50f6a65d41bebedfa45b633d3923 2023-02-09 17:54:12,036 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-945bcbc2d2524784ab5e366f1561f6dd 2023-02-09 17:54:12,058 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23] to archive 2023-02-09 17:54:12,059 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:12,061 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,062 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,064 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,065 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,067 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-31235553b25746508cbd40994e4f5327 2023-02-09 17:54:12,068 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-efd00090578b4832b1e213915cf032ec 2023-02-09 17:54:12,069 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/testSplitTableRegionUnevenDaughter=d08cc38916d551c13ac0de49fa30546d-56487b1a99354c748abb2b0622aded23 2023-02-09 17:54:12,074 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:54:12,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:12,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6. 2023-02-09 17:54:12,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4b2902b446be90fa1b555fe9a2d595e6: Waiting for close lock at 1675965252005Running coprocessor pre-close hooks at 1675965252005Disabling compacts and flushes for region at 1675965252005Disabling writes for close at 1675965252005Writing region close event to WAL at 1675965252071 (+66 ms)Running coprocessor post-close hooks at 1675965252075 (+4 ms)Closed at 1675965252076 (+1 ms) 2023-02-09 17:54:12,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:12,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:12,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1d292a7367afe6611400f38936e4376b, disabling compactions & flushes 2023-02-09 17:54:12,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:12,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:12,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. after waiting 0 ms 2023-02-09 17:54:12,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:12,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=4b2902b446be90fa1b555fe9a2d595e6, regionState=CLOSED 2023-02-09 17:54:12,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965252079"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965252079"}]},"ts":"1675965252079"} 2023-02-09 17:54:12,079 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8-bottom] to archive 2023-02-09 17:54:12,080 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:12,084 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/e80a0faa1b0b4485a4794021f605fd58.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-09 17:54:12,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 4b2902b446be90fa1b555fe9a2d595e6, server=jenkins-hbase4.apache.org,44623,1675965244508 in 229 msec 2023-02-09 17:54:12,086 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/80d70edb70c84e1a8d61154e79b81cd7.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=4b2902b446be90fa1b555fe9a2d595e6, UNASSIGN in 238 msec 2023-02-09 17:54:12,087 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/801a9c67b5c2440b981e34eae88f6aa5.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,088 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/79da2298a331464b8d3deeb34a23c8b8.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,090 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586-bottom] to archive 2023-02-09 17:54:12,091 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:12,092 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/c6656382ab014377a27090706eeb4ba4.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,093 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/fba301782a3a4b96a88a607dd8b39643.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,095 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/addbdbbc7743470590b0a13701163363.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,096 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/1a8ac0dd06fd4e90a9ef3987654aa586.d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:54:12,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:12,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b. 2023-02-09 17:54:12,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1d292a7367afe6611400f38936e4376b: Waiting for close lock at 1675965252078Running coprocessor pre-close hooks at 1675965252078Disabling compacts and flushes for region at 1675965252078Disabling writes for close at 1675965252078Writing region close event to WAL at 1675965252097 (+19 ms)Running coprocessor post-close hooks at 1675965252101 (+4 ms)Closed at 1675965252102 (+1 ms) 2023-02-09 17:54:12,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:12,105 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=1d292a7367afe6611400f38936e4376b, regionState=CLOSED 2023-02-09 17:54:12,105 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675965252105"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965252105"}]},"ts":"1675965252105"} 2023-02-09 17:54:12,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-09 17:54:12,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 1d292a7367afe6611400f38936e4376b, server=jenkins-hbase4.apache.org,44623,1675965244508 in 248 msec 2023-02-09 17:54:12,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-09 17:54:12,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d292a7367afe6611400f38936e4376b, UNASSIGN in 262 msec 2023-02-09 17:54:12,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965252112"}]},"ts":"1675965252112"} 2023-02-09 17:54:12,114 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-09 17:54:12,115 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-09 17:54:12,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 290 msec 2023-02-09 17:54:12,148 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:12,148 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:12,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 17:54:12,150 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-09 17:54:12,151 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:12,151 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:12,154 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,163 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,164 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,165 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:12,165 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:12,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 17:54:12,172 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,172 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:12,172 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:12,175 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/recovered.edits] 2023-02-09 17:54:12,175 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/recovered.edits] 2023-02-09 17:54:12,175 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/recovered.edits] 2023-02-09 17:54:12,179 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/98342e15793747ef80b4d093df133e3b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf1/98342e15793747ef80b4d093df133e3b 2023-02-09 17:54:12,179 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/7b597d6a003a464c9bb2cf6185a26ee8 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf1/7b597d6a003a464c9bb2cf6185a26ee8 2023-02-09 17:54:12,180 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/554c50f6a65d41bebedfa45b633d3923 2023-02-09 17:54:12,182 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/79da2298a331464b8d3deeb34a23c8b8 2023-02-09 17:54:12,183 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/db928ebf5c6c477db915cbe0d7a4cc10 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/cf2/db928ebf5c6c477db915cbe0d7a4cc10 2023-02-09 17:54:12,184 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/801a9c67b5c2440b981e34eae88f6aa5 2023-02-09 17:54:12,185 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1594ecdceae64a35a3c4fd22db76084d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/cf2/1594ecdceae64a35a3c4fd22db76084d 2023-02-09 17:54:12,186 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/80d70edb70c84e1a8d61154e79b81cd7 2023-02-09 17:54:12,188 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b/recovered.edits/90.seqid 2023-02-09 17:54:12,188 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/945bcbc2d2524784ab5e366f1561f6dd 2023-02-09 17:54:12,189 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6/recovered.edits/90.seqid 2023-02-09 17:54:12,189 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d292a7367afe6611400f38936e4376b 2023-02-09 17:54:12,190 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:12,191 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/c1b70d73bde54e0dad02ad3eb42b1bfb 2023-02-09 17:54:12,192 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf1/e80a0faa1b0b4485a4794021f605fd58 2023-02-09 17:54:12,196 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/1a8ac0dd06fd4e90a9ef3987654aa586 2023-02-09 17:54:12,197 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/31235553b25746508cbd40994e4f5327 2023-02-09 17:54:12,199 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/56487b1a99354c748abb2b0622aded23 2023-02-09 17:54:12,201 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/addbdbbc7743470590b0a13701163363 2023-02-09 17:54:12,202 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/c6656382ab014377a27090706eeb4ba4 2023-02-09 17:54:12,204 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/efd00090578b4832b1e213915cf032ec 2023-02-09 17:54:12,205 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/cf2/fba301782a3a4b96a88a607dd8b39643 2023-02-09 17:54:12,208 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/recovered.edits/85.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d/recovered.edits/85.seqid 2023-02-09 17:54:12,209 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionUnevenDaughter/d08cc38916d551c13ac0de49fa30546d 2023-02-09 17:54:12,209 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 17:54:12,212 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,237 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-09 17:54:12,266 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:12,266 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:12,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 17:54:12,280 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-09 17:54:12,282 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,282 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-09 17:54:12,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965252282"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:12,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965252282"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:12,283 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965252282"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:12,286 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:54:12,286 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d08cc38916d551c13ac0de49fa30546d, NAME => 'testSplitTableRegionUnevenDaughter,,1675965247422.d08cc38916d551c13ac0de49fa30546d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1d292a7367afe6611400f38936e4376b, NAME => 'testSplitTableRegionUnevenDaughter,,1675965249640.1d292a7367afe6611400f38936e4376b.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 4b2902b446be90fa1b555fe9a2d595e6, NAME => 'testSplitTableRegionUnevenDaughter,26,1675965249640.4b2902b446be90fa1b555fe9a2d595e6.', STARTKEY => '26', ENDKEY => ''}] 2023-02-09 17:54:12,286 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-09 17:54:12,286 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965252286"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:12,288 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-09 17:54:12,290 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:12,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 134 msec 2023-02-09 17:54:12,468 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:12,468 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:12,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 17:54:12,470 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-09 17:54:12,488 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 358) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54704 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54750 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:45538 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54762 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54692 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54720 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36824 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:37116 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36968 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:45364 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36814 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54680 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:44623-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:45554 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:37132 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54452 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36882 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=830 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=236 (was 257), ProcessCount=168 (was 168), AvailableMemoryMB=3425 (was 3610) 2023-02-09 17:54:12,502 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=830, MaxFileDescriptor=60000, SystemLoadAverage=236, ProcessCount=168, AvailableMemoryMB=3423 2023-02-09 17:54:12,503 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:12,503 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:12,503 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:12,511 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:54:12,511 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:54:12,513 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:12,515 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:12,516 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:12,518 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,518 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c empty. 2023-02-09 17:54:12,519 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,519 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 17:54:12,530 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:12,531 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 18021c0707f7eea71bc7fa8ec852264c, NAME => 'testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:12,541 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:12,542 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 18021c0707f7eea71bc7fa8ec852264c, disabling compactions & flushes 2023-02-09 17:54:12,542 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,542 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,542 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. after waiting 0 ms 2023-02-09 17:54:12,542 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,542 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,542 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 18021c0707f7eea71bc7fa8ec852264c: Waiting for close lock at 1675965252542Disabling compacts and flushes for region at 1675965252542Disabling writes for close at 1675965252542Writing region close event to WAL at 1675965252542Closed at 1675965252542 2023-02-09 17:54:12,545 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:12,546 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965252545"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965252545"}]},"ts":"1675965252545"} 2023-02-09 17:54:12,548 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:12,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:12,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965252549"}]},"ts":"1675965252549"} 2023-02-09 17:54:12,551 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-09 17:54:12,554 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:12,555 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:12,555 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:12,555 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:12,555 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:12,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, ASSIGN}] 2023-02-09 17:54:12,558 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, ASSIGN 2023-02-09 17:54:12,559 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:54:12,709 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:12,710 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=18021c0707f7eea71bc7fa8ec852264c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:12,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965252710"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965252710"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965252710"}]},"ts":"1675965252710"} 2023-02-09 17:54:12,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:12,866 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:12,866 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 17:54:12,869 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40182, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 17:54:12,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 18021c0707f7eea71bc7fa8ec852264c, NAME => 'testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:12,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:12,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:12,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,875 INFO [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,877 DEBUG [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1 2023-02-09 17:54:12,877 DEBUG [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1 2023-02-09 17:54:12,877 INFO [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18021c0707f7eea71bc7fa8ec852264c columnFamilyName cf1 2023-02-09 17:54:12,878 INFO [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] regionserver.HStore(310): Store=18021c0707f7eea71bc7fa8ec852264c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:12,878 INFO [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,880 DEBUG [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2 2023-02-09 17:54:12,880 DEBUG [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2 2023-02-09 17:54:12,882 INFO [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18021c0707f7eea71bc7fa8ec852264c columnFamilyName cf2 2023-02-09 17:54:12,882 INFO [StoreOpener-18021c0707f7eea71bc7fa8ec852264c-1] regionserver.HStore(310): Store=18021c0707f7eea71bc7fa8ec852264c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:12,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:12,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:12,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:12,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 18021c0707f7eea71bc7fa8ec852264c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66665292, jitterRate=-0.006609737873077393}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:12,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 18021c0707f7eea71bc7fa8ec852264c: Running coprocessor pre-open hook at 1675965252874Writing region info on filesystem at 1675965252874Initializing all the Stores at 1675965252875 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965252875Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965252875Cleaning up temporary data from old regions at 1675965252885 (+10 ms)Cleaning up detritus from prior splits at 1675965252888 (+3 ms)Running coprocessor post-open hooks at 1675965252895 (+7 ms)Region opened successfully at 1675965252895 2023-02-09 17:54:12,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c., pid=27, masterSystemTime=1675965252866 2023-02-09 17:54:12,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:12,901 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=18021c0707f7eea71bc7fa8ec852264c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:12,902 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965252901"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965252901"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965252901"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965252901"}]},"ts":"1675965252901"} 2023-02-09 17:54:12,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-09 17:54:12,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 in 191 msec 2023-02-09 17:54:12,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-09 17:54:12,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, ASSIGN in 350 msec 2023-02-09 17:54:12,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:12,910 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965252910"}]},"ts":"1675965252910"} 2023-02-09 17:54:12,911 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-09 17:54:12,913 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:12,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 401 msec 2023-02-09 17:54:13,017 DEBUG [Listener at localhost/34581] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 17:54:13,020 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40196, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 17:54:13,027 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,027 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,028 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:13,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 17:54:13,031 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 17:54:13,031 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,032 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 17:54:13,032 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 17:54:13,032 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,032 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,035 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,035 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,035 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,035 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,035 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,036 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,037 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,037 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,037 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,038 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,038 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,038 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,038 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,038 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,038 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,039 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,039 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:13,039 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:13,039 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,039 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,039 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,039 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,039 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. started... 2023-02-09 17:54:13,040 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:54:13,040 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,041 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,041 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,041 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,042 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,042 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,052 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/7915076fd6f64da19b218dadd63a585a 2023-02-09 17:54:13,071 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/56f81edaae2049faadb154c4477e8b29 2023-02-09 17:54:13,077 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/7915076fd6f64da19b218dadd63a585a as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a 2023-02-09 17:54:13,082 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:13,083 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/56f81edaae2049faadb154c4477e8b29 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 2023-02-09 17:54:13,087 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:13,088 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 48ms, sequenceid=6, compaction requested=false 2023-02-09 17:54:13,088 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-09 17:54:13,089 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 18021c0707f7eea71bc7fa8ec852264c: 2023-02-09 17:54:13,089 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,089 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:13,089 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:13,089 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,089 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,089 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,091 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,091 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,091 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,091 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 17:54:13,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@242d3261[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:13,094 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 17:54:13,094 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,096 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,096 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:13,097 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,098 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,100 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:54:13,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 17:54:13,104 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 17:54:13,104 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:13,104 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:13,104 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 17:54:13,104 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,106 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,107 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,108 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,109 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,110 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,110 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,113 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 17:54:13,114 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:13,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,114 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,115 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:13,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,115 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,115 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:13,115 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,116 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,116 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,116 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,116 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,216 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:13,216 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,216 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:13,235 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,235 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,236 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:13,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:13,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 17:54:13,237 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 17:54:13,238 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,238 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 17:54:13,238 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 17:54:13,239 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,239 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,242 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,242 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,242 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,242 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,242 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,242 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:13,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,245 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:13,246 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,246 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. started... 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,246 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,247 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:13,248 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,249 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,249 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,249 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,249 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,249 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,259 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/b0ed447d21124eeca9d7933685117afd 2023-02-09 17:54:13,276 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/ca12294ea3b64707b8f899d3eb4ce5ac 2023-02-09 17:54:13,282 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/b0ed447d21124eeca9d7933685117afd as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd 2023-02-09 17:54:13,287 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:13,288 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/ca12294ea3b64707b8f899d3eb4ce5ac as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac 2023-02-09 17:54:13,292 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:13,293 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 47ms, sequenceid=19, compaction requested=false 2023-02-09 17:54:13,293 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 18021c0707f7eea71bc7fa8ec852264c: 2023-02-09 17:54:13,293 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,296 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,296 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,296 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 17:54:13,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@afc5f6[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:13,298 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 17:54:13,298 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,300 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:13,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,300 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,301 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,301 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,301 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:13,300 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:13,301 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:13,302 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,302 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,302 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,306 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,306 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,306 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,306 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,307 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,307 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:54:13,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 17:54:13,311 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 17:54:13,311 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:13,311 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:13,311 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 17:54:13,311 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,315 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,314 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,315 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,315 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,317 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,318 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,318 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,322 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,322 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,322 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 17:54:13,323 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:13,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,321 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,323 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,322 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,323 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,323 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:13,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,323 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,323 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:13,324 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,324 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,324 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,324 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,424 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:13,424 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,424 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:13,444 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,444 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,445 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:13,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:13,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 17:54:13,447 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 17:54:13,447 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,448 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 17:54:13,448 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 17:54:13,448 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,448 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,449 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,449 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,449 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,449 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,449 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,450 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,451 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,451 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,451 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,451 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,451 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,451 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,452 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,452 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,453 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,452 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:13,453 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:13,453 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,453 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. started... 2023-02-09 17:54:13,454 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:13,455 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,455 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,455 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,455 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,455 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,455 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,465 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/27ab7b165e6a4a738c38363a6ca331ef 2023-02-09 17:54:13,479 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/9c2445379bbd4254b5e946721f39b529 2023-02-09 17:54:13,484 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/27ab7b165e6a4a738c38363a6ca331ef as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef 2023-02-09 17:54:13,489 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:13,490 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/9c2445379bbd4254b5e946721f39b529 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 2023-02-09 17:54:13,495 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:13,496 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 42ms, sequenceid=32, compaction requested=true 2023-02-09 17:54:13,496 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 18021c0707f7eea71bc7fa8ec852264c: 2023-02-09 17:54:13,496 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,496 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:13,496 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:13,496 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,497 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,497 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,498 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,498 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,499 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,499 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 17:54:13,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6455e128[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:13,501 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 17:54:13,501 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,503 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,503 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:13,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,503 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,503 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,504 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,504 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:13,503 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:13,503 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,504 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:13,505 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,505 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,505 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,505 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,505 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,506 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,507 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,507 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,507 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,507 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:54:13,512 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 17:54:13,512 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 17:54:13,512 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:13,512 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:13,512 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 17:54:13,512 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:13,513 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,513 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,514 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,514 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,515 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,515 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,517 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,518 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,518 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,522 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:13,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,524 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,525 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,525 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 17:54:13,526 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 17:54:13,526 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:13,526 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-09 17:54:13,624 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:13,625 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,625 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:13,644 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,644 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,645 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:13,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:13,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 17:54:13,647 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 17:54:13,647 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,648 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 17:54:13,648 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 17:54:13,648 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,648 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,650 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,650 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,650 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,650 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,650 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,650 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,655 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,654 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,655 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,655 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:13,656 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:13,656 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,656 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,656 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,657 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,657 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. started... 2023-02-09 17:54:13,657 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,657 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:13,661 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,662 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,662 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,662 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,662 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,662 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,668 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/9c8f24f55185457b88b46ed27e8291dd 2023-02-09 17:54:13,682 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/a1868bffe28442668c7442e6ad53b5dd 2023-02-09 17:54:13,687 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/9c8f24f55185457b88b46ed27e8291dd as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd 2023-02-09 17:54:13,692 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:13,693 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/a1868bffe28442668c7442e6ad53b5dd as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd 2023-02-09 17:54:13,698 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:13,699 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 42ms, sequenceid=45, compaction requested=true 2023-02-09 17:54:13,699 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 18021c0707f7eea71bc7fa8ec852264c: 2023-02-09 17:54:13,699 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,700 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:13,700 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:13,700 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,700 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,700 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,702 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,702 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 17:54:13,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@382a0303[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:13,704 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 17:54:13,704 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,706 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,706 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,706 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,707 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,707 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:13,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:13,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:13,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,707 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:13,708 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,708 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,708 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,709 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,709 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,709 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,709 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,710 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:54:13,715 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 17:54:13,715 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:13,715 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:13,715 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 17:54:13,715 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 17:54:13,715 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:13,717 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,717 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,717 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,717 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,717 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,718 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,717 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,718 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,718 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,719 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,721 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,721 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,721 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,725 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,725 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 17:54:13,725 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:13,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,725 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:13,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:13,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,826 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:13,826 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,826 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:13,851 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:13,851 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:13,852 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:13,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:13,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 17:54:13,854 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 17:54:13,854 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,854 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 17:54:13,854 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 17:54:13,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,857 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,858 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,860 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:13,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:13,860 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:13,860 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:13,860 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:13,861 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,862 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,863 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,863 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,863 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,863 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:13,864 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,864 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. started... 2023-02-09 17:54:13,864 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:13,878 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/90fb37c21d574b538a4561482843fe5b 2023-02-09 17:54:13,893 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d 2023-02-09 17:54:13,898 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/90fb37c21d574b538a4561482843fe5b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b 2023-02-09 17:54:13,903 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:13,904 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d 2023-02-09 17:54:13,908 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:13,909 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 45ms, sequenceid=58, compaction requested=true 2023-02-09 17:54:13,910 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 18021c0707f7eea71bc7fa8ec852264c: 2023-02-09 17:54:13,910 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:13,910 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:13,910 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:13,910 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,910 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:13,910 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:13,916 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,916 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,916 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,916 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:13,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 17:54:13,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7559805d[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:13,918 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 17:54:13,918 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,919 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,919 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,919 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:13,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,920 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,920 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:13,921 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,921 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,921 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,921 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,921 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,922 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,922 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,922 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:54:13,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 17:54:13,925 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 17:54:13,925 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:13,925 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:13,925 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 17:54:13,925 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,928 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,928 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:13,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,928 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:13,930 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:13,930 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,931 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,936 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,936 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 17:54:13,937 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:13,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:13,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:13,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,936 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:13,937 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,937 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:13,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,937 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:13,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:13,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:13,937 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:13,937 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:13,938 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,938 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:13,938 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:13,938 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:13,938 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,038 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:14,038 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:14,038 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:14,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:14,056 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:14,056 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:14,057 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:14,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:14,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 17:54:14,059 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 17:54:14,059 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:14,059 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 17:54:14,059 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 17:54:14,060 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,060 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,061 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:14,061 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:14,061 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:14,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:14,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:14,061 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:14,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:14,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:14,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:14,062 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,062 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 17:54:14,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:14,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:14,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 17:54:14,063 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:14,064 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:14,064 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:14,064 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. started... 2023-02-09 17:54:14,064 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:14,065 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,065 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:14,075 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/27b3c4539cc549f7ac12b3350baf6cbe 2023-02-09 17:54:14,089 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/1de1ac39e28346bb9372b56a688b1080 2023-02-09 17:54:14,094 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/27b3c4539cc549f7ac12b3350baf6cbe as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe 2023-02-09 17:54:14,099 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:14,100 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/1de1ac39e28346bb9372b56a688b1080 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 2023-02-09 17:54:14,106 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:14,108 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 43ms, sequenceid=71, compaction requested=true 2023-02-09 17:54:14,108 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 18021c0707f7eea71bc7fa8ec852264c: 2023-02-09 17:54:14,108 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:14,108 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:14,108 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:14,108 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,108 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 17:54:14,108 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 17:54:14,111 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,111 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:14,111 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,111 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:14,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:14,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:14,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:14,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 17:54:14,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ba76b75[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:14,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 17:54:14,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,114 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,114 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:14,115 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,115 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:14,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:14,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:14,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 17:54:14,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:14,117 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 17:54:14,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:14,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:14,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:14,118 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:14,118 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 17:54:14,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:14,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:14,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:14,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:14,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:14,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:54:14,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 17:54:14,121 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 17:54:14,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:14,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:14,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 17:54:14,121 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:14,123 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:14,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:14,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:14,123 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:14,123 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,124 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:14,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,124 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:14,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:14,124 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:14,125 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:14,126 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 17:54:14,127 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,127 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 17:54:14,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:14,130 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 17:54:14,130 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:14,130 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:14,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:14,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:14,129 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:14,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:14,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:14,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:14,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:14,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:14,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,130 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:14,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 17:54:14,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:14,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 17:54:14,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:14,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:14,231 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:14,231 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:14,231 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:14,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:14,496 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 17:54:14,498 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:14,499 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:14,502 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:14,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:14,750 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,751 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:14,751 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:14,751 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:14,753 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-09 17:54:14,754 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:14,754 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:14,754 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:14,754 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:14,755 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:14,758 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 17:54:14,760 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:14,761 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:14,761 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:14,762 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:14,762 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:14,763 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:14,763 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:14,764 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:14,764 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:14,765 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,765 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:14,765 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,765 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,765 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,765 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 17:54:14,766 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:14,767 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:14,768 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965284768 2023-02-09 17:54:14,769 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:14,769 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,773 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:14,773 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:14,774 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:14,774 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965314774 2023-02-09 17:54:14,774 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965374774 2023-02-09 17:54:14,774 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:14,774 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:14,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:14,777 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:14,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:14,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:15,024 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,024 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:15,024 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:15,024 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:15,025 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:15,025 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:15,025 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:15,025 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:15,025 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:15,026 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:15,028 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 17:54:15,028 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:15,029 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:15,029 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:15,029 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:15,030 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,030 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:15,030 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,030 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,031 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:15,031 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,032 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,032 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,032 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,032 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,032 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:54:15,033 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:15,033 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:15,034 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965285034 2023-02-09 17:54:15,035 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:15,035 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,038 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:15,039 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:15,039 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:15,039 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965315039 2023-02-09 17:54:15,039 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965375039 2023-02-09 17:54:15,039 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:15,039 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:15,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN}] 2023-02-09 17:54:15,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:15,042 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:15,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:15,289 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,290 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:15,290 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:15,290 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:15,291 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:15,291 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:15,291 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:15,291 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:15,291 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:15,292 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:15,293 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 17:54:15,294 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:15,294 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:15,294 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:15,295 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:15,295 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,296 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:15,296 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,296 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,296 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:15,297 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,297 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,297 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,297 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,297 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,297 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:15,298 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:15,298 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:15,298 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965285298 2023-02-09 17:54:15,299 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:15,300 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,302 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:15,303 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:15,303 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:15,303 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965315303 2023-02-09 17:54:15,303 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965375303 2023-02-09 17:54:15,303 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:15,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN}] 2023-02-09 17:54:15,304 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:15,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,306 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:15,307 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965255306"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965255306"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965255306"}]},"ts":"1675965255306"} 2023-02-09 17:54:15,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:15,308 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:15,308 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:15,308 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:15,553 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:15,554 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:15,554 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:15,555 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:15,556 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:15,557 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 17:54:15,557 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:15,557 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:15,557 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:15,558 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:15,558 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:15,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:15,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,560 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,560 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,561 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,561 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,561 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,561 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,561 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:15,561 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:15,562 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:15,562 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c to restore RIT 2023-02-09 17:54:15,562 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965285562 2023-02-09 17:54:15,563 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:15,563 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,566 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:15,566 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:15,566 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:15,566 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965315566 2023-02-09 17:54:15,566 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965375566 2023-02-09 17:54:15,566 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:15,567 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:15,567 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965255567"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965255567"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965255567"}]},"ts":"1675965255567"} 2023-02-09 17:54:15,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:15,568 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:15,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:15,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 18021c0707f7eea71bc7fa8ec852264c, disabling compactions & flushes 2023-02-09 17:54:15,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:15,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:15,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. after waiting 0 ms 2023-02-09 17:54:15,722 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:15,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 18021c0707f7eea71bc7fa8ec852264c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:54:15,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/26823f21aeac453d89d64a643281e37f 2023-02-09 17:54:15,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/ec7311e783ad448cb2962c7c31a78100 2023-02-09 17:54:15,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf1/26823f21aeac453d89d64a643281e37f as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f 2023-02-09 17:54:15,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:54:15,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/.tmp/cf2/ec7311e783ad448cb2962c7c31a78100 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 2023-02-09 17:54:15,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:54:15,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 53ms, sequenceid=82, compaction requested=true 2023-02-09 17:54:15,799 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:54:15,799 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:15,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. 2023-02-09 17:54:15,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 18021c0707f7eea71bc7fa8ec852264c: Waiting for close lock at 1675965255722Running coprocessor pre-close hooks at 1675965255722Disabling compacts and flushes for region at 1675965255722Disabling writes for close at 1675965255722Obtaining lock to block concurrent updates at 1675965255722Preparing flush snapshotting stores in 18021c0707f7eea71bc7fa8ec852264c at 1675965255722Finished memstore snapshotting testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675965255723 (+1 ms)Flushing stores of testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. at 1675965255723Flushing 18021c0707f7eea71bc7fa8ec852264c/cf1: creating writer at 1675965255724 (+1 ms)Flushing 18021c0707f7eea71bc7fa8ec852264c/cf1: appending metadata at 1675965255726 (+2 ms)Flushing 18021c0707f7eea71bc7fa8ec852264c/cf1: closing flushed file at 1675965255726Flushing 18021c0707f7eea71bc7fa8ec852264c/cf2: creating writer at 1675965255740 (+14 ms)Flushing 18021c0707f7eea71bc7fa8ec852264c/cf2: appending metadata at 1675965255742 (+2 ms)Flushing 18021c0707f7eea71bc7fa8ec852264c/cf2: closing flushed file at 1675965255742Flushing 18021c0707f7eea71bc7fa8ec852264c/cf1: reopening flushed file at 1675965255755 (+13 ms)Flushing 18021c0707f7eea71bc7fa8ec852264c/cf2: reopening flushed file at 1675965255768 (+13 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 18021c0707f7eea71bc7fa8ec852264c in 53ms, sequenceid=82, compaction requested=true at 1675965255775 (+7 ms)Writing region close event to WAL at 1675965255795 (+20 ms)Running coprocessor post-close hooks at 1675965255799 (+4 ms)Closed at 1675965255800 (+1 ms) 2023-02-09 17:54:15,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:15,803 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED 2023-02-09 17:54:15,803 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965255803"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965255803"}]},"ts":"1675965255803"} 2023-02-09 17:54:15,804 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:15,804 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:15,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:15,805 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:15,817 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:15,817 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:15,818 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:15,819 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:15,819 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 17:54:15,819 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 17:54:15,819 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:15,820 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:15,820 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:15,820 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:15,821 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,821 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:15,821 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,821 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:15,822 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:15,822 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,822 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,822 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:15,822 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,823 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,823 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,823 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:15,823 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,823 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,823 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:15,823 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:15,824 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:15,824 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:15,824 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c to restore RIT 2023-02-09 17:54:15,824 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965285824 2023-02-09 17:54:15,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:15,825 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:15,827 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:15,828 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:15,828 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:15,828 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965315828 2023-02-09 17:54:15,828 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965375828 2023-02-09 17:54:15,828 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:15,828 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:15,828 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965255828"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965255828"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965255828"}]},"ts":"1675965255828"} 2023-02-09 17:54:15,830 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:15,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-09 17:54:15,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 in 262 msec 2023-02-09 17:54:15,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:15,833 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:15,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:15,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:16,078 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,078 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:16,078 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:16,079 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:16,080 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:16,080 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:16,080 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:16,080 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:16,080 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:16,081 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:16,082 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:16,082 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 17:54:16,082 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:16,082 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:16,083 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:16,083 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:16,083 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:16,084 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:16,084 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:16,084 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:16,085 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:16,085 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:16,086 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,086 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:16,086 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:16,086 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:16,086 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,086 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,086 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,086 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:16,086 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:16,086 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:16,087 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:16,088 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:16,088 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c to restore RIT 2023-02-09 17:54:16,088 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965286088 2023-02-09 17:54:16,089 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:16,089 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:16,091 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:16,092 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:16,092 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:16,092 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965316092 2023-02-09 17:54:16,092 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965376092 2023-02-09 17:54:16,092 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:16,092 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:16,094 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:16,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-09 17:54:16,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN in 791 msec 2023-02-09 17:54:16,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:16,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:16,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:16,342 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,342 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:16,342 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:16,342 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:16,343 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:16,343 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:16,343 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:16,343 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:16,343 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:16,344 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:16,345 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:16,345 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:16,345 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:16,345 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:16,346 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:16,346 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:16,346 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:16,347 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:16,347 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:16,347 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:16,347 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:16,348 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:16,348 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,348 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:16,348 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:16,348 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:16,349 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:16,349 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:16,349 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:16,349 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:16,350 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:16,350 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965286350 2023-02-09 17:54:16,352 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:16,352 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,354 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:16,355 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:16,355 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:16,355 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965316355 2023-02-09 17:54:16,355 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965376355 2023-02-09 17:54:16,355 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:16,355 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,355 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,355 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,360 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:16,370 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=18021c0707f7eea71bc7fa8ec852264c, threads=14 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,371 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,372 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,486 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,486 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,495 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,512 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,516 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,519 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,523 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,523 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,524 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,524 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,525 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,528 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,528 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,770 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:54:16,841 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:16,842 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 18021c0707f7eea71bc7fa8ec852264c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:54:16,887 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:16,887 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:16,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:16,888 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:17,105 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:17,106 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:17,106 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:17,107 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:17,108 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 17:54:17,108 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:17,108 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:17,109 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:17,109 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:17,110 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:17,110 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:17,110 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,111 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:17,111 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,111 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,112 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:17,113 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,113 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:17,113 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:17,113 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:17,114 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,114 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,114 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,114 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:17,114 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:17,114 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 17:54:17,115 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:17,115 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:17,116 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965287116 2023-02-09 17:54:17,117 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,117 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:17,120 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:17,120 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:17,120 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:17,120 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965317120 2023-02-09 17:54:17,120 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965377120 2023-02-09 17:54:17,120 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:17,130 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=18021c0707f7eea71bc7fa8ec852264c, threads=14 2023-02-09 17:54:17,132 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,132 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,132 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,132 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,132 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,132 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,133 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,133 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,133 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,133 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,133 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,134 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,134 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,134 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,353 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,365 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,366 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,368 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,370 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,373 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,377 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,377 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,388 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,391 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,391 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,391 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,394 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,395 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef for region: 18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:17,395 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 18021c0707f7eea71bc7fa8ec852264c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:54:17,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:17,438 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:54:17,440 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:54:17,440 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:17,440 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:17,440 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:17,621 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:17,621 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:17,622 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:17,623 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 17:54:17,623 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:17,623 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:17,624 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:17,624 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:17,624 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:17,624 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:17,625 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,625 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:17,625 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,625 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,626 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:17,626 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,626 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:17,626 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:17,626 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:17,627 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,627 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,627 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,627 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:17,627 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:17,627 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:17,628 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:17,628 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:17,628 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965287628 2023-02-09 17:54:17,629 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:17,630 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,632 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:17,633 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:17,633 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:17,633 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965317633 2023-02-09 17:54:17,633 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965377633 2023-02-09 17:54:17,633 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:17,636 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:17,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:17,638 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:17,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:17,883 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:17,883 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:17,883 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:17,883 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:17,884 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:17,884 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:17,884 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:17,884 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:17,884 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:17,885 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:17,886 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 17:54:17,886 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:17,886 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:17,886 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:17,886 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:17,887 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:17,887 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:17,887 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,887 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:17,888 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,888 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:17,888 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:17,888 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,889 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN 2023-02-09 17:54:17,889 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:17,889 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:17,889 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,889 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,889 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,889 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=18021c0707f7eea71bc7fa8ec852264c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:17,889 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 18021c0707f7eea71bc7fa8ec852264c, server=jenkins-hbase4.apache.org,43923,1675965244585 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:17,889 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:17,890 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:17,890 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:17,891 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965287891 2023-02-09 17:54:17,891 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:17,892 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:17,894 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:17,894 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=18021c0707f7eea71bc7fa8ec852264c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43923,1675965244585, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585, openSeqNum=2 2023-02-09 17:54:17,894 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:17,894 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965317894 2023-02-09 17:54:17,894 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965377894 2023-02-09 17:54:17,894 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:17,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965257896"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675965257896"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675965257896"}]},"ts":"1675965257896"} 2023-02-09 17:54:17,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965257896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965257896"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965257896"}]},"ts":"1675965257896"} 2023-02-09 17:54:17,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965257896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965257896"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965257896"}]},"ts":"1675965257896"} 2023-02-09 17:54:17,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, ASSIGN}] 2023-02-09 17:54:17,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, ASSIGN 2023-02-09 17:54:17,905 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:54:17,905 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, ASSIGN 2023-02-09 17:54:17,906 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:54:18,055 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:54:18,056 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=a39cddc4336f63004ea998d92908d13f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,057 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965258056"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965258056"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965258056"}]},"ts":"1675965258056"} 2023-02-09 17:54:18,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure a39cddc4336f63004ea998d92908d13f, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:18,062 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=95d83923843793eee3680b2585b92089, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,062 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965258062"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965258062"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965258062"}]},"ts":"1675965258062"} 2023-02-09 17:54:18,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 95d83923843793eee3680b2585b92089, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:18,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a39cddc4336f63004ea998d92908d13f, NAME => 'testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 17:54:18,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,216 INFO [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,217 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1 2023-02-09 17:54:18,217 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1 2023-02-09 17:54:18,218 INFO [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a39cddc4336f63004ea998d92908d13f columnFamilyName cf1 2023-02-09 17:54:18,233 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f-bottom 2023-02-09 17:54:18,238 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef-bottom 2023-02-09 17:54:18,244 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe-bottom 2023-02-09 17:54:18,250 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a-bottom 2023-02-09 17:54:18,257 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b-bottom 2023-02-09 17:54:18,266 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd-bottom 2023-02-09 17:54:18,272 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd-bottom 2023-02-09 17:54:18,272 INFO [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(310): Store=a39cddc4336f63004ea998d92908d13f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:18,272 INFO [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,273 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2 2023-02-09 17:54:18,273 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2 2023-02-09 17:54:18,274 INFO [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a39cddc4336f63004ea998d92908d13f columnFamilyName cf2 2023-02-09 17:54:18,288 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080-bottom 2023-02-09 17:54:18,294 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29-bottom 2023-02-09 17:54:18,299 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d-bottom 2023-02-09 17:54:18,303 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529-bottom 2023-02-09 17:54:18,308 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd-bottom 2023-02-09 17:54:18,314 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac-bottom 2023-02-09 17:54:18,318 DEBUG [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100-bottom 2023-02-09 17:54:18,319 INFO [StoreOpener-a39cddc4336f63004ea998d92908d13f-1] regionserver.HStore(310): Store=a39cddc4336f63004ea998d92908d13f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:18,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:18,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a39cddc4336f63004ea998d92908d13f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71964675, jitterRate=0.07235722243785858}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:18,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a39cddc4336f63004ea998d92908d13f: Running coprocessor pre-open hook at 1675965258215Writing region info on filesystem at 1675965258215Initializing all the Stores at 1675965258216 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965258216Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965258216Cleaning up temporary data from old regions at 1675965258321 (+105 ms)Cleaning up detritus from prior splits at 1675965258322 (+1 ms)Running coprocessor post-open hooks at 1675965258325 (+3 ms)Region opened successfully at 1675965258325 2023-02-09 17:54:18,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f., pid=33, masterSystemTime=1675965258211 2023-02-09 17:54:18,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:18,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:18,327 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:18,328 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:18,328 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,328 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1912): a39cddc4336f63004ea998d92908d13f/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:18,328 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,328 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1912): a39cddc4336f63004ea998d92908d13f/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:18,328 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a39cddc4336f63004ea998d92908d13f/cf1 in testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,328 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a39cddc4336f63004ea998d92908d13f/cf2 in testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,328 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/.tmp, totalSize=34.9 K 2023-02-09 17:54:18,328 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/.tmp, totalSize=34.9 K 2023-02-09 17:54:18,329 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965253020 2023-02-09 17:54:18,329 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965253020 2023-02-09 17:54:18,330 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965253218 2023-02-09 17:54:18,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 95d83923843793eee3680b2585b92089, NAME => 'testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 17:54:18,330 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965253218 2023-02-09 17:54:18,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:18,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=a39cddc4336f63004ea998d92908d13f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:18,331 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965258330"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965258330"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965258330"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965258330"}]},"ts":"1675965258330"} 2023-02-09 17:54:18,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,331 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965253427 2023-02-09 17:54:18,331 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965253427 2023-02-09 17:54:18,332 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965253627 2023-02-09 17:54:18,332 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965253627 2023-02-09 17:54:18,332 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965253829 2023-02-09 17:54:18,333 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965253829 2023-02-09 17:54:18,333 INFO [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,333 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965254040 2023-02-09 17:54:18,333 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965254040 2023-02-09 17:54:18,334 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965254233 2023-02-09 17:54:18,334 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965254233 2023-02-09 17:54:18,335 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1 2023-02-09 17:54:18,335 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1 2023-02-09 17:54:18,336 INFO [StoreOpener-95d83923843793eee3680b2585b92089-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95d83923843793eee3680b2585b92089 columnFamilyName cf1 2023-02-09 17:54:18,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-09 17:54:18,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure a39cddc4336f63004ea998d92908d13f, server=jenkins-hbase4.apache.org,43923,1675965244585 in 274 msec 2023-02-09 17:54:18,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, ASSIGN in 436 msec 2023-02-09 17:54:18,354 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] throttle.PressureAwareThroughputController(145): a39cddc4336f63004ea998d92908d13f#cf2#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:18,355 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f-top 2023-02-09 17:54:18,359 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] throttle.PressureAwareThroughputController(145): a39cddc4336f63004ea998d92908d13f#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:18,365 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef-top 2023-02-09 17:54:18,377 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe-top 2023-02-09 17:54:18,383 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a-top 2023-02-09 17:54:18,392 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b-top 2023-02-09 17:54:18,397 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd-top 2023-02-09 17:54:18,398 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/.tmp/cf2/8c7083d45639444a8ec2e401d17f7d75 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/8c7083d45639444a8ec2e401d17f7d75 2023-02-09 17:54:18,400 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/.tmp/cf1/bcb8a974cbaf4dd687ec584a953856b5 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/bcb8a974cbaf4dd687ec584a953856b5 2023-02-09 17:54:18,406 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd-top 2023-02-09 17:54:18,406 INFO [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(310): Store=95d83923843793eee3680b2585b92089/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:18,406 INFO [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,407 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2 2023-02-09 17:54:18,407 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2 2023-02-09 17:54:18,408 INFO [StoreOpener-95d83923843793eee3680b2585b92089-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95d83923843793eee3680b2585b92089 columnFamilyName cf2 2023-02-09 17:54:18,409 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a39cddc4336f63004ea998d92908d13f/cf2 of a39cddc4336f63004ea998d92908d13f into 8c7083d45639444a8ec2e401d17f7d75(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:18,410 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a39cddc4336f63004ea998d92908d13f: 2023-02-09 17:54:18,410 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f., storeName=a39cddc4336f63004ea998d92908d13f/cf2, priority=9, startTime=1675965258326; duration=0sec 2023-02-09 17:54:18,410 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:18,410 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a39cddc4336f63004ea998d92908d13f/cf1 of a39cddc4336f63004ea998d92908d13f into bcb8a974cbaf4dd687ec584a953856b5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:18,410 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a39cddc4336f63004ea998d92908d13f: 2023-02-09 17:54:18,410 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f., storeName=a39cddc4336f63004ea998d92908d13f/cf1, priority=9, startTime=1675965258326; duration=0sec 2023-02-09 17:54:18,410 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:18,423 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080-top 2023-02-09 17:54:18,428 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29-top 2023-02-09 17:54:18,432 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d-top 2023-02-09 17:54:18,435 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529-top 2023-02-09 17:54:18,439 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd-top 2023-02-09 17:54:18,443 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac-top 2023-02-09 17:54:18,447 DEBUG [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100-top 2023-02-09 17:54:18,447 INFO [StoreOpener-95d83923843793eee3680b2585b92089-1] regionserver.HStore(310): Store=95d83923843793eee3680b2585b92089/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:18,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:18,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 95d83923843793eee3680b2585b92089 2023-02-09 17:54:18,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 95d83923843793eee3680b2585b92089; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67798545, jitterRate=0.010277047753334045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:18,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 95d83923843793eee3680b2585b92089: Running coprocessor pre-open hook at 1675965258331Writing region info on filesystem at 1675965258331Initializing all the Stores at 1675965258332 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965258332Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965258332Cleaning up temporary data from old regions at 1675965258449 (+117 ms)Cleaning up detritus from prior splits at 1675965258449Running coprocessor post-open hooks at 1675965258452 (+3 ms)Region opened successfully at 1675965258452 2023-02-09 17:54:18,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089., pid=34, masterSystemTime=1675965258211 2023-02-09 17:54:18,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:18,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:18,454 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:18,454 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:18,454 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,454 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1912): 95d83923843793eee3680b2585b92089/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:18,454 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,454 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 95d83923843793eee3680b2585b92089/cf1 in testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,454 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1912): 95d83923843793eee3680b2585b92089/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:18,454 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 95d83923843793eee3680b2585b92089/cf2 in testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,454 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f-top] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/.tmp, totalSize=34.9 K 2023-02-09 17:54:18,455 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100-top] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/.tmp, totalSize=34.9 K 2023-02-09 17:54:18,455 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965253020 2023-02-09 17:54:18,455 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965253020 2023-02-09 17:54:18,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:18,456 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965253218 2023-02-09 17:54:18,456 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965253218 2023-02-09 17:54:18,456 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965253427 2023-02-09 17:54:18,456 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=95d83923843793eee3680b2585b92089, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,457 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965253427 2023-02-09 17:54:18,457 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965258456"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965258456"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965258456"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965258456"}]},"ts":"1675965258456"} 2023-02-09 17:54:18,457 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965253627 2023-02-09 17:54:18,457 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965253627 2023-02-09 17:54:18,457 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675965253829 2023-02-09 17:54:18,458 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675965253829 2023-02-09 17:54:18,458 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675965254040 2023-02-09 17:54:18,458 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675965254040 2023-02-09 17:54:18,459 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965254233 2023-02-09 17:54:18,459 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965254233 2023-02-09 17:54:18,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-09 17:54:18,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 95d83923843793eee3680b2585b92089, server=jenkins-hbase4.apache.org,43923,1675965244585 in 395 msec 2023-02-09 17:54:18,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-09 17:54:18,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, ASSIGN in 562 msec 2023-02-09 17:54:18,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 in 3.9670 sec 2023-02-09 17:54:18,475 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] throttle.PressureAwareThroughputController(145): 95d83923843793eee3680b2585b92089#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:18,477 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 95d83923843793eee3680b2585b92089#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:18,491 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/.tmp/cf2/bbdd868c38a04e8096e6e55509bc3621 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/bbdd868c38a04e8096e6e55509bc3621 2023-02-09 17:54:18,493 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/.tmp/cf1/c679bba1374e49fd87418c9782313d0a as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/c679bba1374e49fd87418c9782313d0a 2023-02-09 17:54:18,499 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 95d83923843793eee3680b2585b92089/cf1 of 95d83923843793eee3680b2585b92089 into c679bba1374e49fd87418c9782313d0a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:18,499 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 95d83923843793eee3680b2585b92089/cf2 of 95d83923843793eee3680b2585b92089 into bbdd868c38a04e8096e6e55509bc3621(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:18,499 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 95d83923843793eee3680b2585b92089: 2023-02-09 17:54:18,499 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 95d83923843793eee3680b2585b92089: 2023-02-09 17:54:18,499 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089., storeName=95d83923843793eee3680b2585b92089/cf1, priority=9, startTime=1675965258453; duration=0sec 2023-02-09 17:54:18,499 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089., storeName=95d83923843793eee3680b2585b92089/cf2, priority=9, startTime=1675965258454; duration=0sec 2023-02-09 17:54:18,499 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:18,499 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:18,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43923] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:40196 deadline: 1675965318646, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c. is not online on jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,758 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:54:18,766 INFO [Listener at localhost/34581] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 17:54:18,769 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:54:18,792 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:18,792 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:18,792 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:18,793 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:18,794 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:18,794 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-09 17:54:18,794 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-09 17:54:18,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:18,798 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:18,798 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:18,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 17:54:18,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965258799"}]},"ts":"1675965258799"} 2023-02-09 17:54:18,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-09 17:54:18,809 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-09 17:54:18,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, UNASSIGN}] 2023-02-09 17:54:18,811 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, UNASSIGN 2023-02-09 17:54:18,812 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=95d83923843793eee3680b2585b92089, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,812 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965258812"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965258812"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965258812"}]},"ts":"1675965258812"} 2023-02-09 17:54:18,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 95d83923843793eee3680b2585b92089, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:18,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, UNASSIGN 2023-02-09 17:54:18,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a39cddc4336f63004ea998d92908d13f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:18,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965258817"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965258817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965258817"}]},"ts":"1675965258817"} 2023-02-09 17:54:18,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure a39cddc4336f63004ea998d92908d13f, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:18,899 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:18,899 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:18,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 17:54:18,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:18,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a39cddc4336f63004ea998d92908d13f, disabling compactions & flushes 2023-02-09 17:54:18,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. after waiting 0 ms 2023-02-09 17:54:18,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:18,979 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f-bottom] to archive 2023-02-09 17:54:18,980 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:18,982 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,984 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,985 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,986 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,987 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,988 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,989 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,991 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100-bottom] to archive 2023-02-09 17:54:18,992 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:18,994 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,995 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,996 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,997 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,998 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:18,999 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,000 DEBUG [StoreCloser-testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:54:19,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:19,005 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f. 2023-02-09 17:54:19,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a39cddc4336f63004ea998d92908d13f: Waiting for close lock at 1675965258966Running coprocessor pre-close hooks at 1675965258966Disabling compacts and flushes for region at 1675965258966Disabling writes for close at 1675965258966Writing region close event to WAL at 1675965259002 (+36 ms)Running coprocessor post-close hooks at 1675965259005 (+3 ms)Closed at 1675965259005 2023-02-09 17:54:19,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:19,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 95d83923843793eee3680b2585b92089 2023-02-09 17:54:19,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 95d83923843793eee3680b2585b92089, disabling compactions & flushes 2023-02-09 17:54:19,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:19,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:19,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. after waiting 0 ms 2023-02-09 17:54:19,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:19,008 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a39cddc4336f63004ea998d92908d13f, regionState=CLOSED 2023-02-09 17:54:19,008 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965259008"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965259008"}]},"ts":"1675965259008"} 2023-02-09 17:54:19,009 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f-top] to archive 2023-02-09 17:54:19,009 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:19,011 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-09 17:54:19,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure a39cddc4336f63004ea998d92908d13f, server=jenkins-hbase4.apache.org,43923,1675965244585 in 191 msec 2023-02-09 17:54:19,013 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a39cddc4336f63004ea998d92908d13f, UNASSIGN in 202 msec 2023-02-09 17:54:19,014 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,015 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,017 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,018 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,019 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,021 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100-top] to archive 2023-02-09 17:54:19,022 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:19,023 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,025 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,026 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,027 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,028 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,029 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,030 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:54:19,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:19,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089. 2023-02-09 17:54:19,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 95d83923843793eee3680b2585b92089: Waiting for close lock at 1675965259007Running coprocessor pre-close hooks at 1675965259007Disabling compacts and flushes for region at 1675965259007Disabling writes for close at 1675965259007Writing region close event to WAL at 1675965259031 (+24 ms)Running coprocessor post-close hooks at 1675965259035 (+4 ms)Closed at 1675965259035 2023-02-09 17:54:19,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 95d83923843793eee3680b2585b92089 2023-02-09 17:54:19,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=95d83923843793eee3680b2585b92089, regionState=CLOSED 2023-02-09 17:54:19,037 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965259037"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965259037"}]},"ts":"1675965259037"} 2023-02-09 17:54:19,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-09 17:54:19,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 95d83923843793eee3680b2585b92089, server=jenkins-hbase4.apache.org,43923,1675965244585 in 225 msec 2023-02-09 17:54:19,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-09 17:54:19,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=95d83923843793eee3680b2585b92089, UNASSIGN in 231 msec 2023-02-09 17:54:19,044 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965259044"}]},"ts":"1675965259044"} 2023-02-09 17:54:19,045 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-09 17:54:19,047 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-09 17:54:19,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 253 msec 2023-02-09 17:54:19,101 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,101 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 17:54:19,102 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-09 17:54:19,102 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,103 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,103 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-09 17:54:19,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:19,106 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:19,107 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,107 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,107 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:19,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 17:54:19,114 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,114 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:19,114 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089 2023-02-09 17:54:19,116 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/recovered.edits] 2023-02-09 17:54:19,117 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/recovered.edits] 2023-02-09 17:54:19,117 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/recovered.edits] 2023-02-09 17:54:19,121 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/.regioninfo to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/.regioninfo 2023-02-09 17:54:19,121 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/.regioninfo to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/.regioninfo 2023-02-09 17:54:19,121 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/26823f21aeac453d89d64a643281e37f 2023-02-09 17:54:19,123 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27ab7b165e6a4a738c38363a6ca331ef 2023-02-09 17:54:19,124 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/27b3c4539cc549f7ac12b3350baf6cbe 2023-02-09 17:54:19,124 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,124 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/26823f21aeac453d89d64a643281e37f.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,125 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/7915076fd6f64da19b218dadd63a585a 2023-02-09 17:54:19,126 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,126 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/27ab7b165e6a4a738c38363a6ca331ef.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,127 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,128 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/27b3c4539cc549f7ac12b3350baf6cbe.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,128 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/90fb37c21d574b538a4561482843fe5b 2023-02-09 17:54:19,129 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,129 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/7915076fd6f64da19b218dadd63a585a.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,129 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/9c8f24f55185457b88b46ed27e8291dd 2023-02-09 17:54:19,130 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,131 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf1/b0ed447d21124eeca9d7933685117afd 2023-02-09 17:54:19,131 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/90fb37c21d574b538a4561482843fe5b.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,132 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,140 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,140 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/9c8f24f55185457b88b46ed27e8291dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,141 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/1de1ac39e28346bb9372b56a688b1080 2023-02-09 17:54:19,142 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf1/b0ed447d21124eeca9d7933685117afd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,143 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/56f81edaae2049faadb154c4477e8b29 2023-02-09 17:54:19,144 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,145 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d 2023-02-09 17:54:19,147 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,148 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/1de1ac39e28346bb9372b56a688b1080.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,148 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/9c2445379bbd4254b5e946721f39b529 2023-02-09 17:54:19,149 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,150 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/a1868bffe28442668c7442e6ad53b5dd 2023-02-09 17:54:19,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/56f81edaae2049faadb154c4477e8b29.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,151 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,152 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ca12294ea3b64707b8f899d3eb4ce5ac 2023-02-09 17:54:19,152 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/5e2d525ebf2a47ee9d802d9e20e8ca6d.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,153 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,153 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/cf2/ec7311e783ad448cb2962c7c31a78100 2023-02-09 17:54:19,154 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/9c2445379bbd4254b5e946721f39b529.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,154 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,156 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/95d83923843793eee3680b2585b92089/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,156 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/a1868bffe28442668c7442e6ad53b5dd.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,158 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/recovered.edits/85.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c/recovered.edits/85.seqid 2023-02-09 17:54:19,158 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/ca12294ea3b64707b8f899d3eb4ce5ac.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,159 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,159 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/c679bba1374e49fd87418c9782313d0a to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf1/c679bba1374e49fd87418c9782313d0a 2023-02-09 17:54:19,160 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/a39cddc4336f63004ea998d92908d13f/cf2/ec7311e783ad448cb2962c7c31a78100.18021c0707f7eea71bc7fa8ec852264c 2023-02-09 17:54:19,163 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/bbdd868c38a04e8096e6e55509bc3621 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/cf2/bbdd868c38a04e8096e6e55509bc3621 2023-02-09 17:54:19,163 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/bcb8a974cbaf4dd687ec584a953856b5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf1/bcb8a974cbaf4dd687ec584a953856b5 2023-02-09 17:54:19,166 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/8c7083d45639444a8ec2e401d17f7d75 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/cf2/8c7083d45639444a8ec2e401d17f7d75 2023-02-09 17:54:19,166 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089/recovered.edits/90.seqid 2023-02-09 17:54:19,167 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/95d83923843793eee3680b2585b92089 2023-02-09 17:54:19,170 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f/recovered.edits/90.seqid 2023-02-09 17:54:19,171 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitWithoutPONR/a39cddc4336f63004ea998d92908d13f 2023-02-09 17:54:19,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 17:54:19,174 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:19,177 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-09 17:54:19,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-09 17:54:19,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:19,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-09 17:54:19,180 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965259180"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:19,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965259180"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:19,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965259180"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:19,183 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:54:19,183 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 18021c0707f7eea71bc7fa8ec852264c, NAME => 'testSplitWithoutPONR,,1675965252512.18021c0707f7eea71bc7fa8ec852264c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a39cddc4336f63004ea998d92908d13f, NAME => 'testSplitWithoutPONR,,1675965254498.a39cddc4336f63004ea998d92908d13f.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 95d83923843793eee3680b2585b92089, NAME => 'testSplitWithoutPONR,41,1675965254498.95d83923843793eee3680b2585b92089.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 17:54:19,183 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-09 17:54:19,183 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965259183"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:19,185 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-09 17:54:19,187 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:19,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 84 msec 2023-02-09 17:54:19,208 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,208 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 17:54:19,210 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-09 17:54:19,227 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=403 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35620 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:54750 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:45538 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:58854 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:58842 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35026 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35666 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:58806 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35638 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35664 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:58860 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35652 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:58892 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35640 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:43923-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:58826 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_516925052_15 at /127.0.0.1:35608 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2f5e6fb2-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=853 (was 830) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=280 (was 236) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3300 (was 3423) 2023-02-09 17:54:19,242 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=403, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=280, ProcessCount=168, AvailableMemoryMB=3300 2023-02-09 17:54:19,243 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:19,243 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,244 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,247 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:54:19,248 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:54:19,249 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:19,251 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:19,251 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:19,253 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,254 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb empty. 2023-02-09 17:54:19,254 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,254 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 17:54:19,265 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:19,267 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3371e42e10cd6e1053fe33400fe25afb, NAME => 'testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:19,279 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:19,279 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 3371e42e10cd6e1053fe33400fe25afb, disabling compactions & flushes 2023-02-09 17:54:19,279 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,279 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,279 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. after waiting 0 ms 2023-02-09 17:54:19,279 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,279 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,279 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 3371e42e10cd6e1053fe33400fe25afb: Waiting for close lock at 1675965259279Disabling compacts and flushes for region at 1675965259279Disabling writes for close at 1675965259279Writing region close event to WAL at 1675965259279Closed at 1675965259279 2023-02-09 17:54:19,281 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:19,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965259282"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965259282"}]},"ts":"1675965259282"} 2023-02-09 17:54:19,284 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:19,285 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:19,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965259285"}]},"ts":"1675965259285"} 2023-02-09 17:54:19,286 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 17:54:19,289 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:19,290 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:19,290 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:19,290 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:19,290 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:19,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN}] 2023-02-09 17:54:19,292 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:19,292 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:54:19,443 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:19,444 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,444 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965259444"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965259444"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965259444"}]},"ts":"1675965259444"} 2023-02-09 17:54:19,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:19,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3371e42e10cd6e1053fe33400fe25afb, NAME => 'testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:19,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:19,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:19,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,609 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,611 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1 2023-02-09 17:54:19,611 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1 2023-02-09 17:54:19,611 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3371e42e10cd6e1053fe33400fe25afb columnFamilyName cf1 2023-02-09 17:54:19,612 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(310): Store=3371e42e10cd6e1053fe33400fe25afb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:19,612 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,613 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2 2023-02-09 17:54:19,613 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2 2023-02-09 17:54:19,614 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3371e42e10cd6e1053fe33400fe25afb columnFamilyName cf2 2023-02-09 17:54:19,614 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(310): Store=3371e42e10cd6e1053fe33400fe25afb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:19,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:19,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:19,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:19,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3371e42e10cd6e1053fe33400fe25afb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73165564, jitterRate=0.0902518630027771}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:19,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3371e42e10cd6e1053fe33400fe25afb: Running coprocessor pre-open hook at 1675965259608Writing region info on filesystem at 1675965259608Initializing all the Stores at 1675965259609 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965259609Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965259609Cleaning up temporary data from old regions at 1675965259616 (+7 ms)Cleaning up detritus from prior splits at 1675965259617 (+1 ms)Running coprocessor post-open hooks at 1675965259621 (+4 ms)Region opened successfully at 1675965259621 2023-02-09 17:54:19,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb., pid=43, masterSystemTime=1675965259598 2023-02-09 17:54:19,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,624 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965259624"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965259624"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965259624"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965259624"}]},"ts":"1675965259624"} 2023-02-09 17:54:19,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-09 17:54:19,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 in 180 msec 2023-02-09 17:54:19,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-09 17:54:19,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN in 338 msec 2023-02-09 17:54:19,631 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:19,631 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965259631"}]},"ts":"1675965259631"} 2023-02-09 17:54:19,633 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 17:54:19,640 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:19,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 392 msec 2023-02-09 17:54:19,759 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,759 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,760 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:19,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 17:54:19,762 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 17:54:19,762 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,762 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 17:54:19,762 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:54:19,763 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,763 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,764 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,764 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,764 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,764 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,765 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,765 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:19,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:19,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:19,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:19,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:19,767 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:19,767 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:19,768 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,768 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. started... 2023-02-09 17:54:19,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:19,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:19,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,769 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:19,770 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:19,770 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:19,770 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,770 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:19,770 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:54:19,771 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,771 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,771 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:19,793 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/d7b74e904e3c46d7870efc4e57bb489e 2023-02-09 17:54:19,813 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 2023-02-09 17:54:19,822 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/d7b74e904e3c46d7870efc4e57bb489e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e 2023-02-09 17:54:19,827 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:19,828 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 2023-02-09 17:54:19,834 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:19,835 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 65ms, sequenceid=6, compaction requested=false 2023-02-09 17:54:19,835 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-09 17:54:19,836 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:19,836 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:19,836 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:19,836 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:19,836 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,836 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:19,836 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:19,838 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,838 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:19,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:19,839 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,839 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:19,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:19,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:19,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 17:54:19,842 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:54:19,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40b08788[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:19,842 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,844 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:19,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:19,845 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:19,847 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,847 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:19,847 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:19,847 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:19,847 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:19,848 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:19,848 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:19,848 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:19,850 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:19,850 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:19,850 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,850 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:19,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:19,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:19,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:19,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:19,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:19,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:54:19,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:54:19,855 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 17:54:19,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:19,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:19,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 17:54:19,855 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:19,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,858 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:19,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:19,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,859 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,860 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,862 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:19,862 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:19,862 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,865 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,865 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,865 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:19,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,865 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,866 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 17:54:19,866 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,866 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:19,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,865 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:19,865 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:19,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:19,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:19,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:19,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:19,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:19,867 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:19,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:19,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:19,967 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:19,968 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,968 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:19,987 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:19,987 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:19,988 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:19,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:19,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 17:54:19,990 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 17:54:19,990 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,990 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 17:54:19,990 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:54:19,991 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,991 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,992 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,992 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,992 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:19,992 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:19,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:19,993 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:19,993 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:19,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:19,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:19,995 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:19,995 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:19,995 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,995 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:19,998 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,999 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,999 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,999 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:19,999 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:19,999 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,000 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,000 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,000 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,000 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:20,000 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:20,000 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,000 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. started... 2023-02-09 17:54:20,001 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:20,023 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 2023-02-09 17:54:20,042 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 2023-02-09 17:54:20,047 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 2023-02-09 17:54:20,051 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:20,052 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 2023-02-09 17:54:20,057 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:20,058 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 58ms, sequenceid=19, compaction requested=false 2023-02-09 17:54:20,058 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:20,058 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,059 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:20,059 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:20,059 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,059 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,059 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,061 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,061 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,061 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,061 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 17:54:20,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28eab686[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:20,063 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:54:20,063 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,064 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:20,065 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,065 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,065 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:20,067 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,076 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,076 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,076 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,076 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,076 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,076 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:54:20,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:54:20,078 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 17:54:20,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:20,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:20,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 17:54:20,078 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,081 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,081 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,081 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,083 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,083 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,083 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,087 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,087 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:20,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,087 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,087 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:20,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,188 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:20,188 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,188 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:20,207 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,207 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,208 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:20,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:20,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 17:54:20,210 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 17:54:20,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:54:20,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,212 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,212 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,212 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,213 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,213 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:20,214 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,214 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. started... 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:20,215 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:20,216 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,216 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,217 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,223 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,224 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,224 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,230 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/9da922bf0e1a4e349780610c4026a24e 2023-02-09 17:54:20,250 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/4b61ba19adeb45129373a4dc63b85476 2023-02-09 17:54:20,256 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/9da922bf0e1a4e349780610c4026a24e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e 2023-02-09 17:54:20,260 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:20,261 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/4b61ba19adeb45129373a4dc63b85476 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 2023-02-09 17:54:20,266 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:20,267 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 52ms, sequenceid=32, compaction requested=true 2023-02-09 17:54:20,267 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:20,267 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,267 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:20,267 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:20,267 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,267 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,267 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,270 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,270 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,270 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,270 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 17:54:20,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@118890ae[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:20,272 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:54:20,273 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:20,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,274 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,274 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:20,274 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,275 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,275 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:20,276 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,276 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,276 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,277 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,277 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,277 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,277 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,279 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,279 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,279 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:54:20,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:54:20,283 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 17:54:20,283 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:20,283 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:20,283 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 17:54:20,283 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:20,284 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,284 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,284 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,284 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,284 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,284 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,285 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,285 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,285 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,286 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,288 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,288 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,288 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,294 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,295 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,294 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,295 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,295 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:20,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:20,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,296 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,396 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:20,396 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,396 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:20,412 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,412 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,413 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:20,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:20,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 17:54:20,415 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,415 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,415 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 17:54:20,415 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:54:20,415 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,415 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,418 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,418 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,418 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,418 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,418 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,418 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,419 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,419 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:20,420 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,420 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. started... 2023-02-09 17:54:20,420 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,421 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,422 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,422 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,422 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,430 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/563c50151b83437cb55bf14a6a830c51 2023-02-09 17:54:20,450 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/a05047f0dfa4404ab77e2e754eebd1ed 2023-02-09 17:54:20,455 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/563c50151b83437cb55bf14a6a830c51 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 2023-02-09 17:54:20,461 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:20,462 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/a05047f0dfa4404ab77e2e754eebd1ed as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed 2023-02-09 17:54:20,466 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:20,467 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 47ms, sequenceid=45, compaction requested=true 2023-02-09 17:54:20,467 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:20,467 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,467 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:20,467 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:20,467 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,467 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,467 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,469 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,469 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,469 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,469 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 17:54:20,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21c4bd29[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:20,471 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:54:20,471 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,472 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:20,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,473 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,473 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,473 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:20,475 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,475 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,475 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,475 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,475 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,475 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,476 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,476 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:54:20,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:54:20,479 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 17:54:20,479 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:20,479 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:20,479 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 17:54:20,479 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,480 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,481 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,481 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,481 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,483 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,483 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,484 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,487 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,487 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,487 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,487 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:20,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:20,488 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,489 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,589 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:20,589 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,589 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:20,605 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,605 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,606 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:20,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:20,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 17:54:20,608 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,608 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,608 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 17:54:20,608 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:54:20,608 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,608 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,610 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,610 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,610 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,610 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,610 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,610 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:20,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,612 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,612 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. started... 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,613 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:20,612 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,613 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,615 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,615 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,616 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,616 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,616 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,616 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,631 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/04c87ed577f647aea1a0706c1ea75c35 2023-02-09 17:54:20,644 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/80042ddf137849e99982ec02e7c2dc7c 2023-02-09 17:54:20,648 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/04c87ed577f647aea1a0706c1ea75c35 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 2023-02-09 17:54:20,652 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:20,653 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/80042ddf137849e99982ec02e7c2dc7c as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c 2023-02-09 17:54:20,656 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:20,657 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 44ms, sequenceid=58, compaction requested=true 2023-02-09 17:54:20,658 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:20,658 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,658 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:20,658 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:20,658 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,658 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,658 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,659 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,660 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,660 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,660 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 17:54:20,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@650cf25e[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:20,662 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:54:20,662 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,664 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,664 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,664 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:20,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,665 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,665 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,665 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:20,666 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,666 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,666 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,666 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,669 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,669 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,669 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,669 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,670 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,670 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:54:20,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:54:20,672 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 17:54:20,672 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:20,672 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:20,672 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 17:54:20,672 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,676 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,676 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,678 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,678 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,679 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,681 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,681 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,681 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,681 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,682 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,782 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:20,782 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,782 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:20,802 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,802 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,803 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:20,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:20,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 17:54:20,805 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,805 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,805 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 17:54:20,805 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:54:20,805 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,806 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,807 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,807 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,807 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,807 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,808 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,808 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,809 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,809 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:20,810 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:20,810 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,810 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. started... 2023-02-09 17:54:20,810 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:20,811 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,813 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,813 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,813 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,814 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,814 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,823 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/eec44d966557456b9b6d03f68137965b 2023-02-09 17:54:20,844 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/246bb3a94c6741e199f9b7c2d5a578ba 2023-02-09 17:54:20,851 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/eec44d966557456b9b6d03f68137965b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b 2023-02-09 17:54:20,856 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:20,857 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/246bb3a94c6741e199f9b7c2d5a578ba as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba 2023-02-09 17:54:20,862 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:20,864 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 53ms, sequenceid=71, compaction requested=true 2023-02-09 17:54:20,864 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:20,864 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:20,864 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:20,864 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:20,864 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,864 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 17:54:20,864 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 17:54:20,866 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,866 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,866 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,866 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:20,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 17:54:20,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18af26ff[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:20,868 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:54:20,869 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,870 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:20,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 17:54:20,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:20,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,872 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,872 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,872 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,874 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,874 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 17:54:20,874 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,874 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,874 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,874 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 17:54:20,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:54:20,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:54:20,877 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 17:54:20,877 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:20,877 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:20,877 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 17:54:20,877 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:20,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:20,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:20,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,880 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,881 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:20,881 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:20,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 17:54:20,883 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,883 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,883 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,891 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:20,891 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 17:54:20,892 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:20,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:20,891 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 17:54:20,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:20,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,892 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:20,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:20,893 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,893 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:20,893 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:20,893 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 17:54:20,893 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 17:54:20,993 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:20,993 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:20,993 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:20,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:21,254 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 17:54:21,255 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:21,256 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:21,258 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:21,259 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:21,507 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:21,507 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:21,508 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:21,509 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 17:54:21,510 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:21,510 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:21,510 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:21,510 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:21,511 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:21,511 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:21,511 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:21,511 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:21,511 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:21,512 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:21,512 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:21,512 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:21,512 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:21,513 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,513 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:21,513 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,513 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,513 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,513 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:21,514 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:21,514 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:21,514 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965291514 2023-02-09 17:54:21,514 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:21,516 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,518 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:21,518 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:21,518 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:21,519 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965321519 2023-02-09 17:54:21,519 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965381519 2023-02-09 17:54:21,519 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:21,519 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:21,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:21,521 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:21,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:21,769 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,769 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:21,769 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:21,769 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:21,770 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:21,770 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:21,770 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:21,770 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:21,770 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:21,771 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:21,772 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 17:54:21,772 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:21,773 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:21,773 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:21,773 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:21,773 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:21,774 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:21,774 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:21,774 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:21,774 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:21,775 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:21,775 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:21,775 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:21,775 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:21,775 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,775 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:21,776 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,776 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,776 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,776 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:21,776 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:21,777 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:21,777 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965291777 2023-02-09 17:54:21,779 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:21,779 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,781 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:21,781 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:21,781 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:21,781 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965321781 2023-02-09 17:54:21,781 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965381781 2023-02-09 17:54:21,781 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:21,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:21,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN}] 2023-02-09 17:54:21,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:21,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:21,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:21,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:22,031 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,031 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:22,032 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:22,032 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:22,033 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:22,033 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:22,033 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:22,033 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:22,033 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:22,034 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:22,035 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 17:54:22,035 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:22,035 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:22,036 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:22,036 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:22,036 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,036 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:22,036 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,037 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,037 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,037 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:22,038 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,038 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,038 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:22,039 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,039 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,039 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,039 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,039 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,039 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:22,040 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:22,040 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:22,040 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965292040 2023-02-09 17:54:22,040 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:22,042 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,044 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,044 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,044 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:22,044 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965322044 2023-02-09 17:54:22,044 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965382044 2023-02-09 17:54:22,044 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:22,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN}] 2023-02-09 17:54:22,045 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:22,047 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,048 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,048 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965262048"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965262048"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965262048"}]},"ts":"1675965262048"} 2023-02-09 17:54:22,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:22,050 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:22,050 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:22,050 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:22,294 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,295 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:22,295 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:22,295 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:22,297 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:22,297 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:22,297 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:22,297 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:22,297 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:22,298 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:22,299 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:22,299 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 17:54:22,299 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:22,300 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:22,300 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:22,300 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:22,300 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,301 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:22,301 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,301 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,301 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,301 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:22,302 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,302 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,302 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:22,303 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,303 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,303 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,303 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,303 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,303 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,303 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,303 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,303 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:22,304 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:22,304 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:22,304 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:22,305 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965292305 2023-02-09 17:54:22,305 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:22,306 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,307 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,308 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,308 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:22,308 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965322308 2023-02-09 17:54:22,308 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965382308 2023-02-09 17:54:22,308 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:22,308 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,308 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965262308"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965262308"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965262308"}]},"ts":"1675965262308"} 2023-02-09 17:54:22,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:22,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:22,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:22,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3371e42e10cd6e1053fe33400fe25afb, disabling compactions & flushes 2023-02-09 17:54:22,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:22,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:22,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. after waiting 0 ms 2023-02-09 17:54:22,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:22,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 3371e42e10cd6e1053fe33400fe25afb 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:54:22,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/dd60a494ac964a78bccd44e4f8c75ae3 2023-02-09 17:54:22,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/d90f8828d8784868aa11efebb0102252 2023-02-09 17:54:22,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/dd60a494ac964a78bccd44e4f8c75ae3 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 2023-02-09 17:54:22,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:54:22,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/d90f8828d8784868aa11efebb0102252 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 2023-02-09 17:54:22,506 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:54:22,507 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 43ms, sequenceid=82, compaction requested=true 2023-02-09 17:54:22,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:54:22,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:22,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:22,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3371e42e10cd6e1053fe33400fe25afb: Waiting for close lock at 1675965262464Running coprocessor pre-close hooks at 1675965262464Disabling compacts and flushes for region at 1675965262464Disabling writes for close at 1675965262464Obtaining lock to block concurrent updates at 1675965262464Preparing flush snapshotting stores in 3371e42e10cd6e1053fe33400fe25afb at 1675965262464Finished memstore snapshotting testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675965262464Flushing stores of testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. at 1675965262465 (+1 ms)Flushing 3371e42e10cd6e1053fe33400fe25afb/cf1: creating writer at 1675965262466 (+1 ms)Flushing 3371e42e10cd6e1053fe33400fe25afb/cf1: appending metadata at 1675965262468 (+2 ms)Flushing 3371e42e10cd6e1053fe33400fe25afb/cf1: closing flushed file at 1675965262468Flushing 3371e42e10cd6e1053fe33400fe25afb/cf2: creating writer at 1675965262484 (+16 ms)Flushing 3371e42e10cd6e1053fe33400fe25afb/cf2: appending metadata at 1675965262486 (+2 ms)Flushing 3371e42e10cd6e1053fe33400fe25afb/cf2: closing flushed file at 1675965262486Flushing 3371e42e10cd6e1053fe33400fe25afb/cf1: reopening flushed file at 1675965262497 (+11 ms)Flushing 3371e42e10cd6e1053fe33400fe25afb/cf2: reopening flushed file at 1675965262502 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3371e42e10cd6e1053fe33400fe25afb in 43ms, sequenceid=82, compaction requested=true at 1675965262507 (+5 ms)Writing region close event to WAL at 1675965262509 (+2 ms)Running coprocessor post-close hooks at 1675965262513 (+4 ms)Closed at 1675965262513 2023-02-09 17:54:22,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:22,517 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED 2023-02-09 17:54:22,517 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965262517"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965262517"}]},"ts":"1675965262517"} 2023-02-09 17:54:22,518 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:22,518 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:22,518 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:22,558 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,558 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:22,558 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:22,559 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:22,559 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:22,559 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:22,559 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:22,559 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:22,559 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:22,560 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:22,561 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:22,561 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 17:54:22,561 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 17:54:22,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:22,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:22,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:22,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:22,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:22,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,564 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,564 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:22,564 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,564 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,565 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:22,565 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,566 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,566 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,566 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,566 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,566 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,566 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,566 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,566 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,567 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,567 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:54:22,567 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:22,568 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:22,568 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:22,568 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965292568 2023-02-09 17:54:22,568 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:22,569 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,571 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,571 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,572 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:22,572 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965322572 2023-02-09 17:54:22,572 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965382572 2023-02-09 17:54:22,572 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:22,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,572 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965262572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965262572"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965262572"}]},"ts":"1675965262572"} 2023-02-09 17:54:22,574 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:22,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 17:54:22,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 in 264 msec 2023-02-09 17:54:22,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:22,577 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:22,578 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:22,822 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,822 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:22,822 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:22,822 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:22,823 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:22,823 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:22,823 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:22,824 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:22,824 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:22,824 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:22,825 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:22,825 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 17:54:22,825 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:22,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:22,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:22,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:22,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:22,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:22,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:22,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:22,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:22,829 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:22,829 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,829 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,830 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:22,830 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:22,830 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,830 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,830 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,830 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:22,830 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:22,830 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:22,831 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:22,831 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:22,831 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:22,831 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965292831 2023-02-09 17:54:22,832 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:22,832 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,835 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,835 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:22,835 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:22,835 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965322835 2023-02-09 17:54:22,835 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965382835 2023-02-09 17:54:22,835 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:22,835 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:22,837 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:22,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 17:54:22,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN in 792 msec 2023-02-09 17:54:22,843 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:22,843 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:22,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:22,843 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:23,085 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,086 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:23,086 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:23,086 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:23,087 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:23,087 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:23,087 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:23,087 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:23,087 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:23,088 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:23,089 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:23,089 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:23,089 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:23,089 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:23,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:23,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:23,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:23,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:23,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:23,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:23,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:23,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:23,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:23,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:23,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:23,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:23,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:23,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:23,094 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:23,094 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:23,094 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:23,094 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:54:23,095 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:23,096 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:23,105 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965293105 2023-02-09 17:54:23,106 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,106 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:23,109 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:23,109 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:23,109 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:23,109 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965323109 2023-02-09 17:54:23,109 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965383109 2023-02-09 17:54:23,109 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:23,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,114 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:23,124 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=3371e42e10cd6e1053fe33400fe25afb, threads=14 2023-02-09 17:54:23,125 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,125 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,125 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,128 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,128 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,128 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,128 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,129 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,129 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,130 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,130 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,130 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,136 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,136 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,334 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,350 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,360 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,360 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,366 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,380 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,382 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,383 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,384 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,384 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,386 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,386 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,387 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,389 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,389 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 3371e42e10cd6e1053fe33400fe25afb Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:54:23,428 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:23,429 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:23,429 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:23,610 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:23,610 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:23,611 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:23,612 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 17:54:23,612 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:23,612 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:23,613 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:23,613 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:23,613 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:23,614 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:23,614 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:23,614 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:23,615 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:23,615 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:23,615 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:23,616 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:23,616 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:23,616 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:23,616 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:23,617 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,617 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:23,617 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:23,617 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:23,617 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,617 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,617 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,618 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:23,618 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:23,618 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:54:23,618 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:23,619 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:23,619 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965293619 2023-02-09 17:54:23,621 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:23,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,623 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:23,623 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:23,623 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:23,623 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965323623 2023-02-09 17:54:23,623 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965383623 2023-02-09 17:54:23,623 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:23,632 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=3371e42e10cd6e1053fe33400fe25afb, threads=14 2023-02-09 17:54:23,633 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,633 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,633 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,633 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,633 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,634 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,635 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,635 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,718 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:54:23,822 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,824 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,848 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,853 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,860 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,866 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,872 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,876 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,884 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,889 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,898 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,908 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,909 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,911 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 for region: 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:23,911 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 3371e42e10cd6e1053fe33400fe25afb Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:54:23,976 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:23,987 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/4aabcf7d5345e7c98b9b70b3983de444/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:54:23,990 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/c7f34f4d98b171ae6edb78220c6f35c5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:54:23,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:23,990 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:23,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:23,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:24,123 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,124 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:24,124 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:24,124 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:24,125 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:24,125 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:24,125 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:24,125 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:24,125 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:24,126 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:24,127 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 17:54:24,127 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:24,127 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:24,128 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:24,128 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:24,128 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:24,129 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:24,129 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,129 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:24,129 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,130 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,130 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,130 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:24,130 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,130 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,131 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:24,131 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,131 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:24,131 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:24,133 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:24,133 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,134 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,134 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,134 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,134 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,134 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 17:54:24,134 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:24,135 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:24,135 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965294135 2023-02-09 17:54:24,136 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:24,136 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,139 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,139 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,139 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:24,139 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965324139 2023-02-09 17:54:24,139 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965384139 2023-02-09 17:54:24,139 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:24,142 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:24,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:24,145 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:24,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:24,390 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,390 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:24,390 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:24,390 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:24,391 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:24,392 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:24,392 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:24,392 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:24,392 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:24,393 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:24,394 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 17:54:24,394 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:24,394 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:24,394 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:24,395 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:24,395 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:24,395 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:24,396 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,396 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:24,397 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,397 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,397 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,398 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:24,398 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,398 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,398 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:24,399 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,399 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,399 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,399 DEBUG [Listener at localhost/34581] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,399 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:54:24,400 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:24,400 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:24,401 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965294401 2023-02-09 17:54:24,401 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:24,404 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,406 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,406 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,406 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:24,406 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965324406 2023-02-09 17:54:24,406 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965384406 2023-02-09 17:54:24,406 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-09 17:54:24,407 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:24,420 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:24,420 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:24,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:24,421 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:24,658 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,658 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:24,658 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:24,658 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:24,658 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:24,659 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:24,659 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:24,659 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:24,659 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:24,659 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:24,661 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 17:54:24,661 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:24,661 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:24,661 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:24,662 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:24,662 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:24,662 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:24,662 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,662 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:24,662 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,663 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,663 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,663 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:24,663 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,663 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,664 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:24,664 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,664 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:24,664 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:24,665 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:24,665 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,665 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,665 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,665 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,665 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,665 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:24,665 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:24,666 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:24,666 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965294666 2023-02-09 17:54:24,666 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:24,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,671 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,671 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,671 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:24,671 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965324671 2023-02-09 17:54:24,671 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965384671 2023-02-09 17:54:24,671 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:24,671 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:24,673 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:24,673 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:24,674 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:24,674 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:24,922 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,922 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:24,923 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:24,923 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:24,923 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:24,923 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:24,923 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:24,923 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:24,924 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:24,924 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:24,925 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 17:54:24,925 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:24,925 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:24,925 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:24,926 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:24,926 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:24,926 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:24,926 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,927 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:24,927 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,927 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:24,927 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,927 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:24,928 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,928 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:24,928 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:24,929 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,929 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:24,929 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:24,929 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:24,929 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,929 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,929 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,929 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,929 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:24,929 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:24,930 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:24,930 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:24,931 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965294931 2023-02-09 17:54:24,931 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:24,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:24,934 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,935 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:24,935 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:24,935 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965324935 2023-02-09 17:54:24,935 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965384935 2023-02-09 17:54:24,935 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:24,935 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:24,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:24,937 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:24,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:24,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:25,185 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,185 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:25,185 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:25,186 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:25,187 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:25,187 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:25,187 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:25,187 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:25,187 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:25,188 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:25,189 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 17:54:25,189 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:25,189 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:25,189 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:25,190 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:25,190 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:25,190 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:25,190 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,191 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:25,191 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,191 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,191 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,191 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:25,192 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,192 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,192 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:25,192 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,193 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:25,193 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:25,193 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:25,193 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,193 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,193 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,193 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,193 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,193 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 17:54:25,194 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:25,194 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:25,195 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965295195 2023-02-09 17:54:25,196 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:25,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,199 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,199 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,200 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:25,200 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965325200 2023-02-09 17:54:25,200 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965385200 2023-02-09 17:54:25,200 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:54:25,200 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:25,204 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN] 2023-02-09 17:54:25,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:25,204 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:25,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:25,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:25,450 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,450 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:25,450 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:25,450 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:25,453 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:25,453 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:25,453 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:25,453 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:25,453 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:25,454 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:25,455 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 17:54:25,455 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:54:25,455 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:54:25,455 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 17:54:25,455 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:25,455 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:25,456 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:25,456 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:25,456 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,456 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:25,456 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,456 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,457 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,457 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:25,457 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,457 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,457 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:25,458 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,458 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:25,458 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:25,458 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:25,458 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:25,459 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,459 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,459 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,459 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,459 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,459 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,459 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:25,459 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:25,460 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:25,460 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:25,460 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965295460 2023-02-09 17:54:25,461 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:25,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,463 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:25,465 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,465 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,465 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:25,465 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965325465 2023-02-09 17:54:25,465 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965385465 2023-02-09 17:54:25,465 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:25,465 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:25,470 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:25,470 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:25,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:25,470 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:25,715 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,715 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:25,716 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:25,716 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:25,716 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:25,716 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:25,716 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:25,716 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:25,716 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:25,717 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:25,717 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:54:25,717 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 17:54:25,718 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:25,718 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:25,718 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:25,718 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:25,718 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:25,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:25,720 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,720 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,720 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:25,720 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,720 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:25,720 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:25,720 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,721 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,721 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,721 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,721 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-09 17:54:25,721 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:25,722 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:25,722 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:25,722 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965295722 2023-02-09 17:54:25,723 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:25,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,725 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:25,727 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,727 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,727 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:25,727 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965325727 2023-02-09 17:54:25,727 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965385727 2023-02-09 17:54:25,727 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:25,727 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:25,729 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:25,729 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:25,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:25,729 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:25,977 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:25,978 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:25,978 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:25,979 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:25,979 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 17:54:25,980 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 17:54:25,980 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:25,980 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:25,980 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:25,980 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:25,981 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,981 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:25,981 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,981 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:25,981 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,981 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:25,982 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,982 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:25,982 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:25,982 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,982 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:25,982 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:25,983 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,983 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,983 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,983 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:25,983 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:25,984 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:25,984 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:25,984 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:25,984 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965295984 2023-02-09 17:54:25,985 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:25,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:25,987 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:25,990 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,990 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:25,990 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:25,990 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965325990 2023-02-09 17:54:25,990 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965385990 2023-02-09 17:54:25,990 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-09 17:54:25,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:25,993 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:25,993 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 17:54:25,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:25,993 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:26,240 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,240 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:26,240 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:26,241 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:26,241 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:26,241 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:26,241 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:26,241 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:26,241 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:26,242 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:26,242 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 17:54:26,242 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 17:54:26,243 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:26,243 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:26,243 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:26,243 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:26,243 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:26,244 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:26,244 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:26,244 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:26,244 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:26,244 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:26,244 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:26,245 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:26,245 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:26,245 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,245 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,245 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:26,245 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,245 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,245 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,246 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:54:26,246 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-09 17:54:26,246 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:26,247 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:26,247 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:26,247 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965296247 2023-02-09 17:54:26,249 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:26,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,251 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,253 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:26,253 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:26,253 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:26,253 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965326253 2023-02-09 17:54:26,253 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965386253 2023-02-09 17:54:26,253 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:54:26,253 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:54:26,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 exec-time=4.9980 sec 2023-02-09 17:54:26,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,257 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=true 2023-02-09 17:54:26,407 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:26,408 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:26,408 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965266408"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965266408"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965266408"}]},"ts":"1675965266408"} 2023-02-09 17:54:26,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:26,410 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:54:26,410 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:54:26,410 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:54:26,503 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:26,503 WARN [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:54:26,504 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:54:26,504 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:54:26,505 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:54:26,506 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-09 17:54:26,506 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:54:26,506 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:54:26,507 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:54:26,507 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:54:26,507 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:26,507 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:54:26,507 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:26,508 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:54:26,508 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:26,508 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:54:26,508 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:26,508 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:54:26,508 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:26,509 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:54:26,509 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,509 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:54:26,509 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,509 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 17:54:26,510 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:54:26,510 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:54:26,510 INFO [Listener at localhost/34581] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb to restore RIT 2023-02-09 17:54:26,510 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965296510 2023-02-09 17:54:26,511 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:54:26,511 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:54:26,513 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:26,513 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=3371e42e10cd6e1053fe33400fe25afb, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:54:26,513 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:54:26,513 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965326513 2023-02-09 17:54:26,513 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965386513 2023-02-09 17:54:26,514 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:54:26,514 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:26,514 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965266514"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965266514"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965266514"}]},"ts":"1675965266514"} 2023-02-09 17:54:26,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:26,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:26,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3371e42e10cd6e1053fe33400fe25afb, NAME => 'testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:26,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:26,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:26,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,672 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,673 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1 2023-02-09 17:54:26,673 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1 2023-02-09 17:54:26,674 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3371e42e10cd6e1053fe33400fe25afb columnFamilyName cf1 2023-02-09 17:54:26,680 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 2023-02-09 17:54:26,684 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 2023-02-09 17:54:26,687 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 2023-02-09 17:54:26,690 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e 2023-02-09 17:54:26,694 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e 2023-02-09 17:54:26,697 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 2023-02-09 17:54:26,700 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b 2023-02-09 17:54:26,700 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(310): Store=3371e42e10cd6e1053fe33400fe25afb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:26,700 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,701 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2 2023-02-09 17:54:26,701 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2 2023-02-09 17:54:26,702 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3371e42e10cd6e1053fe33400fe25afb columnFamilyName cf2 2023-02-09 17:54:26,708 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba 2023-02-09 17:54:26,712 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 2023-02-09 17:54:26,715 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c 2023-02-09 17:54:26,719 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 2023-02-09 17:54:26,723 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed 2023-02-09 17:54:26,726 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 2023-02-09 17:54:26,730 DEBUG [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 2023-02-09 17:54:26,730 INFO [StoreOpener-3371e42e10cd6e1053fe33400fe25afb-1] regionserver.HStore(310): Store=3371e42e10cd6e1053fe33400fe25afb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:26,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.splits 2023-02-09 17:54:26,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:26,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:26,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3371e42e10cd6e1053fe33400fe25afb; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68685381, jitterRate=0.023491933941841125}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:26,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3371e42e10cd6e1053fe33400fe25afb: Running coprocessor pre-open hook at 1675965266671Writing region info on filesystem at 1675965266671Initializing all the Stores at 1675965266672 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965266672Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965266672Cleaning up temporary data from old regions at 1675965266732 (+60 ms)Cleaning up detritus from prior splits at 1675965266732Running coprocessor post-open hooks at 1675965266735 (+3 ms)Region opened successfully at 1675965266735 2023-02-09 17:54:26,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb., pid=48, masterSystemTime=1675965266667 2023-02-09 17:54:26,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:26,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:26,738 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:26,738 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:26,739 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 17:54:26,739 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 17:54:26,739 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1912): 3371e42e10cd6e1053fe33400fe25afb/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:26,739 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1912): 3371e42e10cd6e1053fe33400fe25afb/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:26,740 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3371e42e10cd6e1053fe33400fe25afb/cf1 in testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:26,740 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3371e42e10cd6e1053fe33400fe25afb/cf2 in testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:26,740 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp, totalSize=34.9 K 2023-02-09 17:54:26,740 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp, totalSize=34.9 K 2023-02-09 17:54:26,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:26,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:26,740 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting d7b74e904e3c46d7870efc4e57bb489e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965259752 2023-02-09 17:54:26,740 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting e2be3b784fb54ff0bf6c271e1c0e2ca9, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965259752 2023-02-09 17:54:26,740 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:26,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965266740"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965266740"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965266740"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965266740"}]},"ts":"1675965266740"} 2023-02-09 17:54:26,741 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 596f96e79bbe4b3f8acf3a4058bcfd58, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965259970 2023-02-09 17:54:26,741 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 8899a97e7e7f4bfba92ca1f73303d5f3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965259970 2023-02-09 17:54:26,741 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 9da922bf0e1a4e349780610c4026a24e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965260190 2023-02-09 17:54:26,741 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 4b61ba19adeb45129373a4dc63b85476, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965260190 2023-02-09 17:54:26,742 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 563c50151b83437cb55bf14a6a830c51, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965260398 2023-02-09 17:54:26,742 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting a05047f0dfa4404ab77e2e754eebd1ed, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965260398 2023-02-09 17:54:26,742 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 04c87ed577f647aea1a0706c1ea75c35, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965260591 2023-02-09 17:54:26,742 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 80042ddf137849e99982ec02e7c2dc7c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965260591 2023-02-09 17:54:26,742 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting eec44d966557456b9b6d03f68137965b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965260785 2023-02-09 17:54:26,743 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 246bb3a94c6741e199f9b7c2d5a578ba, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965260785 2023-02-09 17:54:26,743 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting dd60a494ac964a78bccd44e4f8c75ae3, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965260994 2023-02-09 17:54:26,743 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting d90f8828d8784868aa11efebb0102252, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965260994 2023-02-09 17:54:26,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-09 17:54:26,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 in 232 msec 2023-02-09 17:54:26,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN in 1.5460 sec 2023-02-09 17:54:26,768 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] throttle.PressureAwareThroughputController(145): 3371e42e10cd6e1053fe33400fe25afb#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:26,768 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3371e42e10cd6e1053fe33400fe25afb#cf1#compaction#50 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:26,787 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf2/6bbc9adb28904aa6bf519708d90e74b4 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/6bbc9adb28904aa6bf519708d90e74b4 2023-02-09 17:54:26,787 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/.tmp/cf1/cea2ade98a6b44a6bc30cabfbbc28e25 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/cea2ade98a6b44a6bc30cabfbbc28e25 2023-02-09 17:54:26,792 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3371e42e10cd6e1053fe33400fe25afb/cf1 of 3371e42e10cd6e1053fe33400fe25afb into cea2ade98a6b44a6bc30cabfbbc28e25(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:26,792 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3371e42e10cd6e1053fe33400fe25afb/cf2 of 3371e42e10cd6e1053fe33400fe25afb into 6bbc9adb28904aa6bf519708d90e74b4(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:26,792 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:26,792 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3371e42e10cd6e1053fe33400fe25afb: 2023-02-09 17:54:26,792 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb., storeName=3371e42e10cd6e1053fe33400fe25afb/cf1, priority=9, startTime=1675965266736; duration=0sec 2023-02-09 17:54:26,792 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb., storeName=3371e42e10cd6e1053fe33400fe25afb/cf2, priority=9, startTime=1675965266737; duration=0sec 2023-02-09 17:54:26,794 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:26,794 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:27,091 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-09 17:54:27,093 DEBUG [Listener at localhost/34581] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-09 17:54:27,095 INFO [Listener at localhost/34581] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 17:54:27,097 INFO [Listener at localhost/34581] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-09 17:54:27,097 INFO [Listener at localhost/34581] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 17:54:27,097 INFO [Listener at localhost/34581] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-09 17:54:27,110 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:27,110 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,111 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,112 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,112 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,112 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-09 17:54:27,113 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-09 17:54:27,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,115 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,115 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 17:54:27,116 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965267116"}]},"ts":"1675965267116"} 2023-02-09 17:54:27,118 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 17:54:27,119 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-09 17:54:27,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN}] 2023-02-09 17:54:27,120 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN 2023-02-09 17:54:27,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:27,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965267121"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965267121"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965267121"}]},"ts":"1675965267121"} 2023-02-09 17:54:27,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:27,216 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,217 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 17:54:27,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:27,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3371e42e10cd6e1053fe33400fe25afb, disabling compactions & flushes 2023-02-09 17:54:27,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:27,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:27,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. after waiting 0 ms 2023-02-09 17:54:27,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:27,275 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3] to archive 2023-02-09 17:54:27,276 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:27,278 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/d7b74e904e3c46d7870efc4e57bb489e 2023-02-09 17:54:27,279 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/596f96e79bbe4b3f8acf3a4058bcfd58 2023-02-09 17:54:27,280 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/9da922bf0e1a4e349780610c4026a24e 2023-02-09 17:54:27,281 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/563c50151b83437cb55bf14a6a830c51 2023-02-09 17:54:27,282 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/04c87ed577f647aea1a0706c1ea75c35 2023-02-09 17:54:27,283 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/eec44d966557456b9b6d03f68137965b 2023-02-09 17:54:27,284 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/dd60a494ac964a78bccd44e4f8c75ae3 2023-02-09 17:54:27,288 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252] to archive 2023-02-09 17:54:27,288 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:27,290 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/e2be3b784fb54ff0bf6c271e1c0e2ca9 2023-02-09 17:54:27,291 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/8899a97e7e7f4bfba92ca1f73303d5f3 2023-02-09 17:54:27,292 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/4b61ba19adeb45129373a4dc63b85476 2023-02-09 17:54:27,292 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/a05047f0dfa4404ab77e2e754eebd1ed 2023-02-09 17:54:27,294 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/80042ddf137849e99982ec02e7c2dc7c 2023-02-09 17:54:27,295 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/246bb3a94c6741e199f9b7c2d5a578ba 2023-02-09 17:54:27,296 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/d90f8828d8784868aa11efebb0102252 2023-02-09 17:54:27,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:54:27,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:27,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb. 2023-02-09 17:54:27,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3371e42e10cd6e1053fe33400fe25afb: Waiting for close lock at 1675965267274Running coprocessor pre-close hooks at 1675965267274Disabling compacts and flushes for region at 1675965267274Disabling writes for close at 1675965267274Writing region close event to WAL at 1675965267297 (+23 ms)Running coprocessor post-close hooks at 1675965267300 (+3 ms)Closed at 1675965267300 2023-02-09 17:54:27,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:27,302 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=3371e42e10cd6e1053fe33400fe25afb, regionState=CLOSED 2023-02-09 17:54:27,302 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965267302"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965267302"}]},"ts":"1675965267302"} 2023-02-09 17:54:27,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 17:54:27,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 3371e42e10cd6e1053fe33400fe25afb, server=jenkins-hbase4.apache.org,35803,1675965244552 in 182 msec 2023-02-09 17:54:27,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 17:54:27,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, UNASSIGN in 186 msec 2023-02-09 17:54:27,308 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965267308"}]},"ts":"1675965267308"} 2023-02-09 17:54:27,309 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 17:54:27,311 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-09 17:54:27,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 199 msec 2023-02-09 17:54:27,418 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,418 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 17:54:27,419 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-09 17:54:27,419 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,419 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,420 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-09 17:54:27,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,422 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,423 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,423 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,423 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 17:54:27,427 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:27,428 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/recovered.edits] 2023-02-09 17:54:27,431 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/cea2ade98a6b44a6bc30cabfbbc28e25 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf1/cea2ade98a6b44a6bc30cabfbbc28e25 2023-02-09 17:54:27,434 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/6bbc9adb28904aa6bf519708d90e74b4 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/cf2/6bbc9adb28904aa6bf519708d90e74b4 2023-02-09 17:54:27,437 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb/recovered.edits/90.seqid 2023-02-09 17:54:27,437 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackAndDoubleExecution/3371e42e10cd6e1053fe33400fe25afb 2023-02-09 17:54:27,437 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 17:54:27,441 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,443 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-09 17:54:27,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-09 17:54:27,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-09 17:54:27,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965267445"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:27,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 17:54:27,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3371e42e10cd6e1053fe33400fe25afb, NAME => 'testRollbackAndDoubleExecution,,1675965259248.3371e42e10cd6e1053fe33400fe25afb.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 17:54:27,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-09 17:54:27,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965267447"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:27,448 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-09 17:54:27,450 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:54:27,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-02-09 17:54:27,524 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,524 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 17:54:27,525 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-09 17:54:27,539 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=403 (was 403), OpenFileDescriptor=853 (was 853), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=370 (was 280) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3288 (was 3300) 2023-02-09 17:54:27,554 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=403, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=370, ProcessCount=168, AvailableMemoryMB=3287 2023-02-09 17:54:27,554 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:27,554 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:27,555 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:27,557 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:54:27,557 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:54:27,558 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:27,560 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:27,561 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:27,562 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,563 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 empty. 2023-02-09 17:54:27,563 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,563 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 17:54:27,573 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:27,574 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2a988393b3c8e714357445e5243b5680, NAME => 'testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:27,584 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:27,584 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 2a988393b3c8e714357445e5243b5680, disabling compactions & flushes 2023-02-09 17:54:27,584 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,584 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,584 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. after waiting 0 ms 2023-02-09 17:54:27,584 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,584 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,584 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 2a988393b3c8e714357445e5243b5680: Waiting for close lock at 1675965267584Disabling compacts and flushes for region at 1675965267584Disabling writes for close at 1675965267584Writing region close event to WAL at 1675965267584Closed at 1675965267584 2023-02-09 17:54:27,586 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:27,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675965267587"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965267587"}]},"ts":"1675965267587"} 2023-02-09 17:54:27,588 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:27,589 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:27,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965267589"}]},"ts":"1675965267589"} 2023-02-09 17:54:27,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-09 17:54:27,594 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:27,595 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:27,595 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:27,595 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:27,595 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:27,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, ASSIGN}] 2023-02-09 17:54:27,597 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, ASSIGN 2023-02-09 17:54:27,597 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:27,748 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:27,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=2a988393b3c8e714357445e5243b5680, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:27,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675965267749"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965267749"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965267749"}]},"ts":"1675965267749"} 2023-02-09 17:54:27,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 2a988393b3c8e714357445e5243b5680, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:27,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2a988393b3c8e714357445e5243b5680, NAME => 'testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:27,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:27,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:27,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,910 INFO [StoreOpener-2a988393b3c8e714357445e5243b5680-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,911 DEBUG [StoreOpener-2a988393b3c8e714357445e5243b5680-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/cf1 2023-02-09 17:54:27,911 DEBUG [StoreOpener-2a988393b3c8e714357445e5243b5680-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/cf1 2023-02-09 17:54:27,912 INFO [StoreOpener-2a988393b3c8e714357445e5243b5680-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a988393b3c8e714357445e5243b5680 columnFamilyName cf1 2023-02-09 17:54:27,912 INFO [StoreOpener-2a988393b3c8e714357445e5243b5680-1] regionserver.HStore(310): Store=2a988393b3c8e714357445e5243b5680/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:27,912 INFO [StoreOpener-2a988393b3c8e714357445e5243b5680-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,913 DEBUG [StoreOpener-2a988393b3c8e714357445e5243b5680-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/cf2 2023-02-09 17:54:27,913 DEBUG [StoreOpener-2a988393b3c8e714357445e5243b5680-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/cf2 2023-02-09 17:54:27,914 INFO [StoreOpener-2a988393b3c8e714357445e5243b5680-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a988393b3c8e714357445e5243b5680 columnFamilyName cf2 2023-02-09 17:54:27,914 INFO [StoreOpener-2a988393b3c8e714357445e5243b5680-1] regionserver.HStore(310): Store=2a988393b3c8e714357445e5243b5680/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:27,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:27,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:27,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:27,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2a988393b3c8e714357445e5243b5680; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68430285, jitterRate=0.019690707325935364}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:27,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2a988393b3c8e714357445e5243b5680: Running coprocessor pre-open hook at 1675965267909Writing region info on filesystem at 1675965267909Initializing all the Stores at 1675965267910 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965267910Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965267910Cleaning up temporary data from old regions at 1675965267916 (+6 ms)Cleaning up detritus from prior splits at 1675965267916Running coprocessor post-open hooks at 1675965267921 (+5 ms)Region opened successfully at 1675965267921 2023-02-09 17:54:27,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680., pid=55, masterSystemTime=1675965267902 2023-02-09 17:54:27,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:27,924 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=2a988393b3c8e714357445e5243b5680, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:27,924 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675965267924"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965267924"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965267924"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965267924"}]},"ts":"1675965267924"} 2023-02-09 17:54:27,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-09 17:54:27,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 2a988393b3c8e714357445e5243b5680, server=jenkins-hbase4.apache.org,44623,1675965244508 in 175 msec 2023-02-09 17:54:27,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-09 17:54:27,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, ASSIGN in 332 msec 2023-02-09 17:54:27,930 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:27,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965267930"}]},"ts":"1675965267930"} 2023-02-09 17:54:27,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-09 17:54:27,933 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:27,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 375 msec 2023-02-09 17:54:28,059 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,060 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,062 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,062 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,062 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, UNASSIGN}] 2023-02-09 17:54:28,064 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, UNASSIGN 2023-02-09 17:54:28,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=2a988393b3c8e714357445e5243b5680, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675965268065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965268065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965268065"}]},"ts":"1675965268065"} 2023-02-09 17:54:28,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 2a988393b3c8e714357445e5243b5680, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:28,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:28,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2a988393b3c8e714357445e5243b5680, disabling compactions & flushes 2023-02-09 17:54:28,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:28,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:28,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. after waiting 0 ms 2023-02-09 17:54:28,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:28,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 17:54:28,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:28,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680. 2023-02-09 17:54:28,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2a988393b3c8e714357445e5243b5680: Waiting for close lock at 1675965268219Running coprocessor pre-close hooks at 1675965268219Disabling compacts and flushes for region at 1675965268219Disabling writes for close at 1675965268220 (+1 ms)Writing region close event to WAL at 1675965268220Running coprocessor post-close hooks at 1675965268224 (+4 ms)Closed at 1675965268224 2023-02-09 17:54:28,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:28,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=2a988393b3c8e714357445e5243b5680, regionState=CLOSED 2023-02-09 17:54:28,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675965268226"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965268226"}]},"ts":"1675965268226"} 2023-02-09 17:54:28,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-09 17:54:28,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 2a988393b3c8e714357445e5243b5680, server=jenkins-hbase4.apache.org,44623,1675965244508 in 160 msec 2023-02-09 17:54:28,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 17:54:28,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2a988393b3c8e714357445e5243b5680, UNASSIGN in 165 msec 2023-02-09 17:54:28,253 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 17:54:28,255 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 17:54:28,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675965268256"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675965268256"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675965268256"}]},"ts":"1675965268256"} 2023-02-09 17:54:28,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268256"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965268256"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965268256"}]},"ts":"1675965268256"} 2023-02-09 17:54:28,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268256"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965268256"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965268256"}]},"ts":"1675965268256"} 2023-02-09 17:54:28,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, ASSIGN}] 2023-02-09 17:54:28,260 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, ASSIGN 2023-02-09 17:54:28,260 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:28,261 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, ASSIGN 2023-02-09 17:54:28,261 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:28,411 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:54:28,412 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=dd3bd80bdb6a1eeb6189e07c0b4193a8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,412 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268412"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965268412"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965268412"}]},"ts":"1675965268412"} 2023-02-09 17:54:28,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure dd3bd80bdb6a1eeb6189e07c0b4193a8, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:28,416 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=130722f0711887fd4e2040a6ee000337, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,416 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268416"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965268416"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965268416"}]},"ts":"1675965268416"} 2023-02-09 17:54:28,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 130722f0711887fd4e2040a6ee000337, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:28,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dd3bd80bdb6a1eeb6189e07c0b4193a8, NAME => 'testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 17:54:28,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:28,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:28,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,570 INFO [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,571 DEBUG [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/cf1 2023-02-09 17:54:28,571 DEBUG [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/cf1 2023-02-09 17:54:28,572 INFO [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd3bd80bdb6a1eeb6189e07c0b4193a8 columnFamilyName cf1 2023-02-09 17:54:28,572 INFO [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] regionserver.HStore(310): Store=dd3bd80bdb6a1eeb6189e07c0b4193a8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:28,572 INFO [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,573 DEBUG [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/cf2 2023-02-09 17:54:28,574 DEBUG [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/cf2 2023-02-09 17:54:28,574 INFO [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd3bd80bdb6a1eeb6189e07c0b4193a8 columnFamilyName cf2 2023-02-09 17:54:28,574 INFO [StoreOpener-dd3bd80bdb6a1eeb6189e07c0b4193a8-1] regionserver.HStore(310): Store=dd3bd80bdb6a1eeb6189e07c0b4193a8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:28,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:28,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened dd3bd80bdb6a1eeb6189e07c0b4193a8; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70501799, jitterRate=0.0505586713552475}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:28,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for dd3bd80bdb6a1eeb6189e07c0b4193a8: Running coprocessor pre-open hook at 1675965268569Writing region info on filesystem at 1675965268569Initializing all the Stores at 1675965268570 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965268570Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965268570Cleaning up temporary data from old regions at 1675965268576 (+6 ms)Cleaning up detritus from prior splits at 1675965268577 (+1 ms)Running coprocessor post-open hooks at 1675965268579 (+2 ms)Region opened successfully at 1675965268579 2023-02-09 17:54:28,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8., pid=61, masterSystemTime=1675965268565 2023-02-09 17:54:28,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 130722f0711887fd4e2040a6ee000337, NAME => 'testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 17:54:28,581 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=dd3bd80bdb6a1eeb6189e07c0b4193a8, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,582 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268581"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965268581"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965268581"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965268581"}]},"ts":"1675965268581"} 2023-02-09 17:54:28,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:28,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:28,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,583 INFO [StoreOpener-130722f0711887fd4e2040a6ee000337-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,584 DEBUG [StoreOpener-130722f0711887fd4e2040a6ee000337-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/cf1 2023-02-09 17:54:28,584 DEBUG [StoreOpener-130722f0711887fd4e2040a6ee000337-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/cf1 2023-02-09 17:54:28,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-09 17:54:28,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure dd3bd80bdb6a1eeb6189e07c0b4193a8, server=jenkins-hbase4.apache.org,44623,1675965244508 in 169 msec 2023-02-09 17:54:28,585 INFO [StoreOpener-130722f0711887fd4e2040a6ee000337-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 130722f0711887fd4e2040a6ee000337 columnFamilyName cf1 2023-02-09 17:54:28,585 INFO [StoreOpener-130722f0711887fd4e2040a6ee000337-1] regionserver.HStore(310): Store=130722f0711887fd4e2040a6ee000337/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:28,585 INFO [StoreOpener-130722f0711887fd4e2040a6ee000337-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, ASSIGN in 326 msec 2023-02-09 17:54:28,586 DEBUG [StoreOpener-130722f0711887fd4e2040a6ee000337-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/cf2 2023-02-09 17:54:28,586 DEBUG [StoreOpener-130722f0711887fd4e2040a6ee000337-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/cf2 2023-02-09 17:54:28,587 INFO [StoreOpener-130722f0711887fd4e2040a6ee000337-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 130722f0711887fd4e2040a6ee000337 columnFamilyName cf2 2023-02-09 17:54:28,587 INFO [StoreOpener-130722f0711887fd4e2040a6ee000337-1] regionserver.HStore(310): Store=130722f0711887fd4e2040a6ee000337/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:28,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:28,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 130722f0711887fd4e2040a6ee000337; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72336897, jitterRate=0.07790376245975494}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:28,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 130722f0711887fd4e2040a6ee000337: Running coprocessor pre-open hook at 1675965268582Writing region info on filesystem at 1675965268582Initializing all the Stores at 1675965268583 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965268583Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965268583Cleaning up temporary data from old regions at 1675965268589 (+6 ms)Cleaning up detritus from prior splits at 1675965268589Running coprocessor post-open hooks at 1675965268592 (+3 ms)Region opened successfully at 1675965268592 2023-02-09 17:54:28,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337., pid=62, masterSystemTime=1675965268565 2023-02-09 17:54:28,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=130722f0711887fd4e2040a6ee000337, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268594"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965268594"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965268594"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965268594"}]},"ts":"1675965268594"} 2023-02-09 17:54:28,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-09 17:54:28,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 130722f0711887fd4e2040a6ee000337, server=jenkins-hbase4.apache.org,44623,1675965244508 in 179 msec 2023-02-09 17:54:28,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-09 17:54:28,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, ASSIGN in 339 msec 2023-02-09 17:54:28,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 in 539 msec 2023-02-09 17:54:28,815 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:54:28,818 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:28,818 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:28,818 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:28,819 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:28,819 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:28,819 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-09 17:54:28,820 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-09 17:54:28,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:28,822 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:28,823 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:28,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 17:54:28,823 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965268823"}]},"ts":"1675965268823"} 2023-02-09 17:54:28,824 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-09 17:54:28,826 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-09 17:54:28,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, UNASSIGN}] 2023-02-09 17:54:28,827 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, UNASSIGN 2023-02-09 17:54:28,828 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=130722f0711887fd4e2040a6ee000337, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,828 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268828"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965268828"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965268828"}]},"ts":"1675965268828"} 2023-02-09 17:54:28,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 130722f0711887fd4e2040a6ee000337, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:28,831 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, UNASSIGN 2023-02-09 17:54:28,831 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=dd3bd80bdb6a1eeb6189e07c0b4193a8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:28,832 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268831"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965268831"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965268831"}]},"ts":"1675965268831"} 2023-02-09 17:54:28,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure dd3bd80bdb6a1eeb6189e07c0b4193a8, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:28,923 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:28,924 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:28,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 17:54:28,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 130722f0711887fd4e2040a6ee000337, disabling compactions & flushes 2023-02-09 17:54:28,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. after waiting 0 ms 2023-02-09 17:54:28,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 17:54:28,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:28,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337. 2023-02-09 17:54:28,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 130722f0711887fd4e2040a6ee000337: Waiting for close lock at 1675965268981Running coprocessor pre-close hooks at 1675965268981Disabling compacts and flushes for region at 1675965268981Disabling writes for close at 1675965268982 (+1 ms)Writing region close event to WAL at 1675965268982Running coprocessor post-close hooks at 1675965268986 (+4 ms)Closed at 1675965268986 2023-02-09 17:54:28,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:28,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing dd3bd80bdb6a1eeb6189e07c0b4193a8, disabling compactions & flushes 2023-02-09 17:54:28,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. after waiting 0 ms 2023-02-09 17:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,988 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=130722f0711887fd4e2040a6ee000337, regionState=CLOSED 2023-02-09 17:54:28,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268988"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965268988"}]},"ts":"1675965268988"} 2023-02-09 17:54:28,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 17:54:28,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 17:54:28,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 130722f0711887fd4e2040a6ee000337, server=jenkins-hbase4.apache.org,44623,1675965244508 in 161 msec 2023-02-09 17:54:28,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:28,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8. 2023-02-09 17:54:28,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for dd3bd80bdb6a1eeb6189e07c0b4193a8: Waiting for close lock at 1675965268987Running coprocessor pre-close hooks at 1675965268987Disabling compacts and flushes for region at 1675965268987Disabling writes for close at 1675965268988 (+1 ms)Writing region close event to WAL at 1675965268989 (+1 ms)Running coprocessor post-close hooks at 1675965268992 (+3 ms)Closed at 1675965268992 2023-02-09 17:54:28,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=130722f0711887fd4e2040a6ee000337, UNASSIGN in 166 msec 2023-02-09 17:54:28,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:28,996 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=dd3bd80bdb6a1eeb6189e07c0b4193a8, regionState=CLOSED 2023-02-09 17:54:28,996 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965268996"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965268996"}]},"ts":"1675965268996"} 2023-02-09 17:54:28,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-09 17:54:28,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure dd3bd80bdb6a1eeb6189e07c0b4193a8, server=jenkins-hbase4.apache.org,44623,1675965244508 in 164 msec 2023-02-09 17:54:29,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-09 17:54:29,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dd3bd80bdb6a1eeb6189e07c0b4193a8, UNASSIGN in 173 msec 2023-02-09 17:54:29,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965269001"}]},"ts":"1675965269001"} 2023-02-09 17:54:29,003 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-09 17:54:29,004 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-09 17:54:29,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 185 msec 2023-02-09 17:54:29,125 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,125 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 17:54:29,126 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-09 17:54:29,126 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,126 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,127 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,129 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,130 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,130 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,130 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 17:54:29,135 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:29,135 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:29,135 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:29,137 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/recovered.edits] 2023-02-09 17:54:29,137 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/recovered.edits] 2023-02-09 17:54:29,138 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/recovered.edits] 2023-02-09 17:54:29,146 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/recovered.edits/7.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8/recovered.edits/7.seqid 2023-02-09 17:54:29,146 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/recovered.edits/7.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337/recovered.edits/7.seqid 2023-02-09 17:54:29,146 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/recovered.edits/4.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680/recovered.edits/4.seqid 2023-02-09 17:54:29,147 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/dd3bd80bdb6a1eeb6189e07c0b4193a8 2023-02-09 17:54:29,147 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/130722f0711887fd4e2040a6ee000337 2023-02-09 17:54:29,147 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionNoStoreFile/2a988393b3c8e714357445e5243b5680 2023-02-09 17:54:29,147 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 17:54:29,149 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,151 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-09 17:54:29,153 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-09 17:54:29,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-09 17:54:29,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965269154"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:29,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965269154"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:29,154 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965269154"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:29,156 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:54:29,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2a988393b3c8e714357445e5243b5680, NAME => 'testSplitTableRegionNoStoreFile,,1675965267558.2a988393b3c8e714357445e5243b5680.', STARTKEY => '', ENDKEY => ''}, {ENCODED => dd3bd80bdb6a1eeb6189e07c0b4193a8, NAME => 'testSplitTableRegionNoStoreFile,,1675965268060.dd3bd80bdb6a1eeb6189e07c0b4193a8.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 130722f0711887fd4e2040a6ee000337, NAME => 'testSplitTableRegionNoStoreFile,41,1675965268060.130722f0711887fd4e2040a6ee000337.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 17:54:29,156 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-09 17:54:29,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965269156"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:29,157 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-09 17:54:29,159 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:54:29,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 32 msec 2023-02-09 17:54:29,231 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,231 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 17:54:29,232 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-09 17:54:29,247 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=407 (was 403) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_668484139_15 at /127.0.0.1:58854 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_668484139_15 at /127.0.0.1:58842 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=853 (was 853), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=340 (was 370), ProcessCount=168 (was 168), AvailableMemoryMB=3284 (was 3287) 2023-02-09 17:54:29,261 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=407, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=168, AvailableMemoryMB=3283 2023-02-09 17:54:29,261 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:29,261 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,261 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,265 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:54:29,265 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:54:29,267 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,268 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:29,269 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:29,271 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,271 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d empty. 2023-02-09 17:54:29,272 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,272 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 17:54:29,287 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:29,289 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4ef9f57187c5579008833c2bab1cfd4d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:29,297 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:29,297 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 4ef9f57187c5579008833c2bab1cfd4d, disabling compactions & flushes 2023-02-09 17:54:29,297 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,297 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,297 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. after waiting 0 ms 2023-02-09 17:54:29,297 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,297 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,298 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 4ef9f57187c5579008833c2bab1cfd4d: Waiting for close lock at 1675965269297Disabling compacts and flushes for region at 1675965269297Disabling writes for close at 1675965269297Writing region close event to WAL at 1675965269297Closed at 1675965269297 2023-02-09 17:54:29,300 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:29,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675965269300"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965269300"}]},"ts":"1675965269300"} 2023-02-09 17:54:29,302 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:29,303 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:29,303 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965269303"}]},"ts":"1675965269303"} 2023-02-09 17:54:29,304 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-09 17:54:29,307 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:29,308 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:29,308 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:29,308 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:29,308 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:29,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, ASSIGN}] 2023-02-09 17:54:29,309 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, ASSIGN 2023-02-09 17:54:29,310 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:29,460 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:29,461 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=4ef9f57187c5579008833c2bab1cfd4d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675965269461"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965269461"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965269461"}]},"ts":"1675965269461"} 2023-02-09 17:54:29,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 4ef9f57187c5579008833c2bab1cfd4d, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:29,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4ef9f57187c5579008833c2bab1cfd4d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:29,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:29,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:29,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,619 INFO [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,620 DEBUG [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1 2023-02-09 17:54:29,620 DEBUG [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1 2023-02-09 17:54:29,620 INFO [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ef9f57187c5579008833c2bab1cfd4d columnFamilyName cf1 2023-02-09 17:54:29,621 INFO [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] regionserver.HStore(310): Store=4ef9f57187c5579008833c2bab1cfd4d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:29,621 INFO [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,622 DEBUG [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2 2023-02-09 17:54:29,623 DEBUG [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2 2023-02-09 17:54:29,623 INFO [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ef9f57187c5579008833c2bab1cfd4d columnFamilyName cf2 2023-02-09 17:54:29,624 INFO [StoreOpener-4ef9f57187c5579008833c2bab1cfd4d-1] regionserver.HStore(310): Store=4ef9f57187c5579008833c2bab1cfd4d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:29,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:29,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:29,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:29,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4ef9f57187c5579008833c2bab1cfd4d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70440944, jitterRate=0.0496518611907959}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:29,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4ef9f57187c5579008833c2bab1cfd4d: Running coprocessor pre-open hook at 1675965269618Writing region info on filesystem at 1675965269618Initializing all the Stores at 1675965269618Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965269618Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965269618Cleaning up temporary data from old regions at 1675965269625 (+7 ms)Cleaning up detritus from prior splits at 1675965269626 (+1 ms)Running coprocessor post-open hooks at 1675965269631 (+5 ms)Region opened successfully at 1675965269631 2023-02-09 17:54:29,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d., pid=71, masterSystemTime=1675965269614 2023-02-09 17:54:29,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,633 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=4ef9f57187c5579008833c2bab1cfd4d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,633 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675965269633"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965269633"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965269633"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965269633"}]},"ts":"1675965269633"} 2023-02-09 17:54:29,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-09 17:54:29,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 4ef9f57187c5579008833c2bab1cfd4d, server=jenkins-hbase4.apache.org,44623,1675965244508 in 172 msec 2023-02-09 17:54:29,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-09 17:54:29,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, ASSIGN in 328 msec 2023-02-09 17:54:29,640 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:29,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965269640"}]},"ts":"1675965269640"} 2023-02-09 17:54:29,641 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-09 17:54:29,643 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:29,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 377 msec 2023-02-09 17:54:29,775 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,775 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,775 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:29,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,777 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:29,777 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,777 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:29,777 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:29,777 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,777 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,779 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,779 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,779 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,779 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,780 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,780 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:29,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:29,781 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:29,781 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:29,781 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:29,782 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:54:29,784 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,784 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,784 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,784 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,784 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:29,784 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:29,792 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/a276e8a48aff4bd5b93e929a730cdaf9 2023-02-09 17:54:29,804 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/cd1ff917f47b4bc995e843e5271b7cc4 2023-02-09 17:54:29,808 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/a276e8a48aff4bd5b93e929a730cdaf9 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9 2023-02-09 17:54:29,811 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:29,812 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/cd1ff917f47b4bc995e843e5271b7cc4 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4 2023-02-09 17:54:29,816 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:54:29,817 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 35ms, sequenceid=6, compaction requested=false 2023-02-09 17:54:29,817 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:29,817 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:29,817 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,817 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:29,817 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:29,817 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,818 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:29,818 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:29,819 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,819 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:29,819 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,819 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:29,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:29,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:29,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:29,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d77d8a4[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:29,821 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:29,821 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,822 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:29,822 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,823 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:29,823 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:29,825 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,825 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:29,825 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:29,825 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:29,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:29,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:29,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:29,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:29,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:29,829 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:29,829 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:29,829 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:29,829 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,829 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:29,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:29,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:29,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:29,832 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,832 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:29,833 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,834 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,834 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,835 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,840 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,840 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,840 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:29,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,841 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:29,840 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,840 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,841 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:29,841 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:29,841 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,840 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:29,841 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:29,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:29,841 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:29,841 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:29,841 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,842 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,942 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:29,942 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,942 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:29,955 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:29,956 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:29,956 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:29,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:29,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,958 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:29,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:29,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:29,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,960 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,960 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,960 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,960 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:29,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:29,960 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,960 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,961 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:29,961 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:29,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:29,962 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:29,963 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:29,963 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:29,963 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,963 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:29,963 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:29,963 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,965 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,965 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,965 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:29,965 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:29,965 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:29,974 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/f369eccc37a04602b8f82f215b9869e1 2023-02-09 17:54:29,985 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/19858728acaf4b7ab41f657dcca07c41 2023-02-09 17:54:29,989 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/f369eccc37a04602b8f82f215b9869e1 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1 2023-02-09 17:54:29,993 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:29,994 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/19858728acaf4b7ab41f657dcca07c41 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41 2023-02-09 17:54:29,997 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:54:29,998 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 35ms, sequenceid=19, compaction requested=false 2023-02-09 17:54:29,998 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:29,998 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:29,998 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:29,998 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:29,998 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:29,999 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:29,999 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,000 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,000 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,000 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,001 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:30,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@495e09d6[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:30,002 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:30,002 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:30,004 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,004 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:30,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:30,004 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:30,005 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,005 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,006 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,006 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,007 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,007 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,007 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,007 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,007 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,007 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:30,011 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:30,011 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:30,011 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:30,011 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:30,011 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,011 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:30,013 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,013 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,013 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,014 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,014 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,014 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,015 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,017 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,020 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,023 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,026 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,026 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,026 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,026 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,027 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,027 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:30,027 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,027 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,027 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:30,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,027 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,127 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:30,128 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,128 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:30,141 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,142 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,142 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:30,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:30,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,144 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:30,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:30,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,146 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,146 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,146 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,147 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,147 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,147 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:30,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,150 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:30,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:30,150 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,150 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:30,150 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:30,152 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,153 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,153 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,153 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,153 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,153 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,161 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/2221f706f4eb48e084a375040d8c4b7f 2023-02-09 17:54:30,177 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/62261d7adc0d4e068ea88563711be661 2023-02-09 17:54:30,181 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/2221f706f4eb48e084a375040d8c4b7f as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f 2023-02-09 17:54:30,185 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:30,186 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/62261d7adc0d4e068ea88563711be661 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661 2023-02-09 17:54:30,189 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:54:30,190 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 40ms, sequenceid=32, compaction requested=true 2023-02-09 17:54:30,190 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:30,190 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,190 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,191 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,195 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,195 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,196 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,196 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:30,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bc29ca1[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:30,197 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:30,197 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:30,199 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,199 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:30,199 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:30,200 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,201 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,201 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:30,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:30,204 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:30,204 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:30,204 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:30,204 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,204 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:30,206 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,206 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,207 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,208 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,208 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,210 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,213 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,213 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,212 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,213 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,213 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,214 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,214 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,214 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,214 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,214 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,214 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,313 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:30,313 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,313 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:30,326 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,326 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,327 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:30,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:30,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,329 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,329 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,329 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:30,329 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:30,329 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,329 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,330 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,330 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,330 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,330 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,331 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,331 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,332 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,332 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:30,333 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:30,333 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,333 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:30,333 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:30,334 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,337 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,337 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,337 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,338 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,338 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,342 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/2c022be3116c4fcc92e166efcebc76d7 2023-02-09 17:54:30,354 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/c70fa46d0e514894b5134c9b6d1453c1 2023-02-09 17:54:30,358 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/2c022be3116c4fcc92e166efcebc76d7 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7 2023-02-09 17:54:30,362 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:30,362 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/c70fa46d0e514894b5134c9b6d1453c1 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1 2023-02-09 17:54:30,366 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:54:30,367 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 34ms, sequenceid=45, compaction requested=true 2023-02-09 17:54:30,367 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:30,367 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,369 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,369 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,369 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,369 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:30,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2abbd3f8[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:30,371 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:30,371 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,372 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,372 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:30,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:30,372 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:30,374 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,374 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,374 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,375 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,375 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,375 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,375 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,375 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,375 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:30,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:30,378 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:30,378 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:30,378 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:30,378 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,378 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,384 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,387 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,387 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,387 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,390 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,390 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:30,390 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,390 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,491 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:30,491 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,491 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:30,502 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,502 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,503 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:30,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:30,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,504 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,504 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,504 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:30,504 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:30,505 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,505 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,507 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,507 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,507 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,507 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,507 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,507 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:30,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:30,509 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,510 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:30,509 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,510 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:30,511 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,511 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,512 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,520 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/f3feaebeb633472db33a173c5bab0485 2023-02-09 17:54:30,532 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/6fd182adbf154d478d23434fdd9aae8b 2023-02-09 17:54:30,536 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/f3feaebeb633472db33a173c5bab0485 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485 2023-02-09 17:54:30,540 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:30,541 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/6fd182adbf154d478d23434fdd9aae8b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b 2023-02-09 17:54:30,544 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:54:30,544 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 34ms, sequenceid=58, compaction requested=true 2023-02-09 17:54:30,545 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:30,545 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,545 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:30,545 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:30,545 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,545 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,545 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,546 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,546 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,546 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,546 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59fa8cf3[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:30,548 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:30,549 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,550 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,550 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,550 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,551 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:30,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:30,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,551 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:30,553 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:30,556 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:30,556 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:30,556 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:30,556 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,556 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:30,558 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,558 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,558 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,558 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,559 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,559 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,559 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,560 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,561 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,562 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,562 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,566 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,566 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,566 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,566 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:30,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,567 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,568 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,667 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:30,667 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,668 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:30,680 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,680 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,680 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:30,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:30,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,682 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,682 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,682 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:30,682 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:30,682 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,683 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,687 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,688 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,688 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:30,689 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:30,689 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,690 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:30,690 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:54:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,698 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/b181a52658254698ab8c1c1a2467398d 2023-02-09 17:54:30,712 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/c16fa9a435bc49b598df6fe3d347d2cf 2023-02-09 17:54:30,716 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/b181a52658254698ab8c1c1a2467398d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d 2023-02-09 17:54:30,719 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:30,720 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/c16fa9a435bc49b598df6fe3d347d2cf as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf 2023-02-09 17:54:30,723 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:54:30,724 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 34ms, sequenceid=71, compaction requested=true 2023-02-09 17:54:30,724 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:30,724 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,724 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:30,724 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:30,724 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,724 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,724 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,726 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,726 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,726 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,726 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:30,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60532ea3[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:30,728 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:30,728 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,729 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,729 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:30,729 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:30,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,730 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:30,731 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,731 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,731 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,732 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:30,736 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:30,736 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:30,736 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:30,736 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,736 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:30,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,738 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,738 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,738 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,739 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,740 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,741 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,741 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,744 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,744 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:30,743 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,744 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:30,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,744 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,745 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,745 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,745 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,745 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,845 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:30,845 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,845 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:30,862 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:30,862 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:30,863 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:30,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:30,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,865 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,865 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,865 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:30,865 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:30,866 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,866 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,867 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,868 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,868 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:30,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,869 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,869 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:30,870 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:30,871 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,871 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:30,871 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-09 17:54:30,873 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,874 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,874 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,874 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,874 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,874 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,882 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:30,886 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:30,895 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:30,899 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:30,900 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:30,903 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:30,904 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 17:54:30,904 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/595488a797174f7fa3165864d2a4f06c as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:30,908 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:30,908 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 17:54:30,910 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 39ms, sequenceid=83, compaction requested=true 2023-02-09 17:54:30,910 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:30,910 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:30,910 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:30,910 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:30,910 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,910 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:30,910 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:30,911 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,911 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,912 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,912 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:30,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:30,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f2b819c[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:30,913 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:30,914 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,914 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,914 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,914 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:30,915 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:30,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:30,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:30,915 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:30,917 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,917 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,917 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,917 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,925 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:30,925 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,925 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:30,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:30,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:30,927 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:30,927 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:30,927 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:30,927 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,927 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:30,933 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,933 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,933 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,933 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:30,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,933 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:30,933 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,934 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,936 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,963 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:30,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:30,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:30,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:30,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:30,963 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:30,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,064 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:31,064 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:31,064 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:31,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:31,072 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:31,072 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:31,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:31,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:31,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,074 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:31,074 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,074 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:31,074 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:31,074 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,074 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,075 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,075 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,075 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,075 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,076 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,076 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:31,077 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,077 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:31,078 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:31,078 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:31,078 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,078 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:31,078 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:31,078 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:31,078 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,078 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:31,078 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 17:54:31,079 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,081 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:31,081 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,081 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:31,087 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:31,090 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:31,097 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:31,100 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:31,100 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:31,104 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:31,104 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 17:54:31,104 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/4a0eb617ed7542cbb9e8af7860c6b57b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:31,107 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:31,107 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 17:54:31,108 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 30ms, sequenceid=91, compaction requested=true 2023-02-09 17:54:31,108 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:31,108 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,108 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:31,108 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:31,108 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,108 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:31,108 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:31,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,110 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:31,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:31,110 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,110 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:31,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:31,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:31,112 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b7ba192[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:31,112 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:31,112 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:31,114 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:31,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:31,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:31,116 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:31,116 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,116 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:31,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:31,117 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:31,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:31,120 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:31,120 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:31,120 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:31,120 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,120 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:31,122 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,122 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,124 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:31,124 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,124 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,130 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:31,130 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:31,130 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,130 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:31,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,131 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,231 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:31,231 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:31,231 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:31,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:31,239 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:31,239 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:31,240 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:54:31,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:54:31,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,241 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:31,241 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,241 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 17:54:31,241 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 17:54:31,242 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,242 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,243 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,243 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,243 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,243 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,243 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,244 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:54:31,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,245 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:31,245 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:31,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,245 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:31,245 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:54:31,246 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,245 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 17:54:31,246 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:31,246 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. started... 2023-02-09 17:54:31,246 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:31,246 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 17:54:31,246 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,247 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,247 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,247 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:31,248 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,248 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:31,259 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:31,263 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:31,271 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:31,276 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:31,277 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/e547245163834ea5bc9b614630ff9903 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:31,281 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:31,281 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 17:54:31,282 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/d301baa024b24ff29a650b619aafeec0 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:31,285 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:31,285 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 17:54:31,286 INFO [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 40ms, sequenceid=99, compaction requested=true 2023-02-09 17:54:31,286 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4ef9f57187c5579008833c2bab1cfd4d: 2023-02-09 17:54:31,286 DEBUG [rs(jenkins-hbase4.apache.org,44623,1675965244508)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,286 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:54:31,286 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:54:31,286 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,286 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 17:54:31,286 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 17:54:31,288 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,288 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:31,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:31,289 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,289 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:54:31,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:31,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:31,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:31,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 17:54:31,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@466b311a[Count = 0] remaining members to acquire global barrier 2023-02-09 17:54:31,290 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 17:54:31,290 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,291 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,291 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,291 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:31,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:31,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,292 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,292 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:54:31,292 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:54:31,293 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:31,293 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,293 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,293 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:31,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:31,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,294 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 17:54:31,294 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:31,294 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,294 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 17:54:31,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,44623,1675965244508': 2023-02-09 17:54:31,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:54:31,297 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 17:54:31,297 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:54:31,297 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:54:31,297 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,297 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:54:31,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:54:31,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:54:31,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,302 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,302 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:54:31,302 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:54:31,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,305 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 17:54:31,305 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,305 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:54:31,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:54:31,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:54:31,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:54:31,305 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:54:31,305 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:54:31,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:54:31,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:54:31,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:31,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:31,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:31,405 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:54:31,406 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:31,406 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:31,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:54:31,409 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,409 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:31,410 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:31,411 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:31,411 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:31,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, UNASSIGN}] 2023-02-09 17:54:31,413 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, UNASSIGN 2023-02-09 17:54:31,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=4ef9f57187c5579008833c2bab1cfd4d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675965271413"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965271413"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965271413"}]},"ts":"1675965271413"} 2023-02-09 17:54:31,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 4ef9f57187c5579008833c2bab1cfd4d, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:31,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4ef9f57187c5579008833c2bab1cfd4d, disabling compactions & flushes 2023-02-09 17:54:31,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. after waiting 0 ms 2023-02-09 17:54:31,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 4ef9f57187c5579008833c2bab1cfd4d 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-09 17:54:31,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:31,579 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:31,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:31,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:31,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:31,594 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:31,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 17:54:31,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.tmp/cf2/e17e66d70e4d4e58921731d22a6adbfa as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:31,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:31,599 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 17:54:31,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 34ms, sequenceid=103, compaction requested=true 2023-02-09 17:54:31,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-09 17:54:31,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:31,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. 2023-02-09 17:54:31,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4ef9f57187c5579008833c2bab1cfd4d: Waiting for close lock at 1675965271566Running coprocessor pre-close hooks at 1675965271566Disabling compacts and flushes for region at 1675965271566Disabling writes for close at 1675965271566Obtaining lock to block concurrent updates at 1675965271566Preparing flush snapshotting stores in 4ef9f57187c5579008833c2bab1cfd4d at 1675965271566Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675965271567 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d. at 1675965271567Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf1: creating writer at 1675965271568 (+1 ms)Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf1: appending metadata at 1675965271570 (+2 ms)Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf1: closing flushed file at 1675965271570Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf2: creating writer at 1675965271580 (+10 ms)Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf2: appending metadata at 1675965271581 (+1 ms)Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf2: closing flushed file at 1675965271581Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf1: reopening flushed file at 1675965271591 (+10 ms)Flushing 4ef9f57187c5579008833c2bab1cfd4d/cf2: reopening flushed file at 1675965271596 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 4ef9f57187c5579008833c2bab1cfd4d in 34ms, sequenceid=103, compaction requested=true at 1675965271600 (+4 ms)Writing region close event to WAL at 1675965271602 (+2 ms)Running coprocessor post-close hooks at 1675965271605 (+3 ms)Closed at 1675965271605 2023-02-09 17:54:31,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=4ef9f57187c5579008833c2bab1cfd4d, regionState=CLOSED 2023-02-09 17:54:31,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675965271606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965271606"}]},"ts":"1675965271606"} 2023-02-09 17:54:31,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-09 17:54:31,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 4ef9f57187c5579008833c2bab1cfd4d, server=jenkins-hbase4.apache.org,44623,1675965244508 in 194 msec 2023-02-09 17:54:31,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 17:54:31,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4ef9f57187c5579008833c2bab1cfd4d, UNASSIGN in 197 msec 2023-02-09 17:54:31,622 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=4ef9f57187c5579008833c2bab1cfd4d, threads=16 2023-02-09 17:54:31,623 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,623 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,623 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,623 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,623 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,623 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,624 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,624 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,624 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,628 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,628 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,628 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,628 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,632 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,632 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,632 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,635 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf, top=false 2023-02-09 17:54:31,637 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:31,637 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7, top=false 2023-02-09 17:54:31,639 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b, top=false 2023-02-09 17:54:31,639 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:31,643 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:31,645 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:31,645 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa, top=true 2023-02-09 17:54:31,648 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:31,650 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1, top=false 2023-02-09 17:54:31,651 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:31,655 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:31,655 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0, top=true 2023-02-09 17:54:31,656 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:31,656 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b, top=true 2023-02-09 17:54:31,657 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:31,657 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d, top=false 2023-02-09 17:54:31,676 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf for child: 1ee04d07a0c83d2cc0c0a9b53cab9a6d, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,677 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa for child: 58cef6fb1a2ed5ac5e50a58a3e6d9dea, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,677 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,678 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,679 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:31,679 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47, top=true 2023-02-09 17:54:31,682 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,683 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,684 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:31,685 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:31,688 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:31,688 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903, top=true 2023-02-09 17:54:31,688 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d for child: 1ee04d07a0c83d2cc0c0a9b53cab9a6d, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,691 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,691 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,694 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485, top=false 2023-02-09 17:54:31,696 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903 for child: 58cef6fb1a2ed5ac5e50a58a3e6d9dea, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,696 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,696 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,699 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:31,702 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:31,702 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5, top=true 2023-02-09 17:54:31,704 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7 for child: 1ee04d07a0c83d2cc0c0a9b53cab9a6d, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,716 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b for child: 1ee04d07a0c83d2cc0c0a9b53cab9a6d, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,717 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485 for child: 1ee04d07a0c83d2cc0c0a9b53cab9a6d, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,742 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,742 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,743 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,743 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:31,748 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1 for child: 1ee04d07a0c83d2cc0c0a9b53cab9a6d, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,749 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,759 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0 for child: 58cef6fb1a2ed5ac5e50a58a3e6d9dea, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,759 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,766 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5 for child: 58cef6fb1a2ed5ac5e50a58a3e6d9dea, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,766 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b for child: 58cef6fb1a2ed5ac5e50a58a3e6d9dea, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,766 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,766 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,768 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,769 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,770 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/.splits/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47 for child: 58cef6fb1a2ed5ac5e50a58a3e6d9dea, parent: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,770 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,776 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,776 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,777 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,777 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,780 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,781 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1 for region: 4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:31,781 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 4ef9f57187c5579008833c2bab1cfd4d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:54:31,824 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 17:54:31,826 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 17:54:31,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675965271828"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675965271828"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675965271828"}]},"ts":"1675965271828"} 2023-02-09 17:54:31,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965271828"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965271828"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965271828"}]},"ts":"1675965271828"} 2023-02-09 17:54:31,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965271828"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965271828"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965271828"}]},"ts":"1675965271828"} 2023-02-09 17:54:31,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, ASSIGN}] 2023-02-09 17:54:31,832 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, ASSIGN 2023-02-09 17:54:31,833 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:31,834 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, ASSIGN 2023-02-09 17:54:31,835 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,44623,1675965244508; forceNewPlan=false, retain=false 2023-02-09 17:54:31,983 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:54:31,985 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=1ee04d07a0c83d2cc0c0a9b53cab9a6d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,985 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965271985"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965271985"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965271985"}]},"ts":"1675965271985"} 2023-02-09 17:54:31,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 1ee04d07a0c83d2cc0c0a9b53cab9a6d, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:31,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=58cef6fb1a2ed5ac5e50a58a3e6d9dea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:31,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965271989"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965271989"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965271989"}]},"ts":"1675965271989"} 2023-02-09 17:54:31,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 58cef6fb1a2ed5ac5e50a58a3e6d9dea, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:32,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1ee04d07a0c83d2cc0c0a9b53cab9a6d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.', STARTKEY => '', ENDKEY => '60'} 2023-02-09 17:54:32,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:32,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:32,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,143 INFO [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,144 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 2023-02-09 17:54:32,144 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 2023-02-09 17:54:32,144 INFO [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ee04d07a0c83d2cc0c0a9b53cab9a6d columnFamilyName cf1 2023-02-09 17:54:32,155 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f-bottom 2023-02-09 17:54:32,159 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9-bottom 2023-02-09 17:54:32,163 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,164 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452-bottom 2023-02-09 17:54:32,169 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1-bottom 2023-02-09 17:54:32,173 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:54:32,173 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7 2023-02-09 17:54:32,177 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d 2023-02-09 17:54:32,180 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485 2023-02-09 17:54:32,180 INFO [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(310): Store=1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:32,181 INFO [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,182 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 2023-02-09 17:54:32,182 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 2023-02-09 17:54:32,182 INFO [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1ee04d07a0c83d2cc0c0a9b53cab9a6d columnFamilyName cf2 2023-02-09 17:54:32,195 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41-bottom 2023-02-09 17:54:32,198 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,200 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c-bottom 2023-02-09 17:54:32,205 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661-bottom 2023-02-09 17:54:32,211 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4-bottom 2023-02-09 17:54:32,215 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b 2023-02-09 17:54:32,218 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf 2023-02-09 17:54:32,222 DEBUG [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1 2023-02-09 17:54:32,222 INFO [StoreOpener-1ee04d07a0c83d2cc0c0a9b53cab9a6d-1] regionserver.HStore(310): Store=1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:32,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:32,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1ee04d07a0c83d2cc0c0a9b53cab9a6d; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69385293, jitterRate=0.033921435475349426}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:32,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1ee04d07a0c83d2cc0c0a9b53cab9a6d: Running coprocessor pre-open hook at 1675965272142Writing region info on filesystem at 1675965272142Initializing all the Stores at 1675965272143 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965272143Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965272143Cleaning up temporary data from old regions at 1675965272225 (+82 ms)Cleaning up detritus from prior splits at 1675965272225Running coprocessor post-open hooks at 1675965272228 (+3 ms)Region opened successfully at 1675965272228 2023-02-09 17:54:32,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d., pid=77, masterSystemTime=1675965272138 2023-02-09 17:54:32,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,230 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:32,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:32,231 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:32,231 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,231 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1912): 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:32,231 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,231 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1912): 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:32,231 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,231 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,231 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp, totalSize=35.0 K 2023-02-09 17:54:32,231 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp, totalSize=35.0 K 2023-02-09 17:54:32,232 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965269770 2023-02-09 17:54:32,232 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965269770 2023-02-09 17:54:32,232 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965269943 2023-02-09 17:54:32,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,233 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965269943 2023-02-09 17:54:32,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,233 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965270129 2023-02-09 17:54:32,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 58cef6fb1a2ed5ac5e50a58a3e6d9dea, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.', STARTKEY => '60', ENDKEY => ''} 2023-02-09 17:54:32,233 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=1ee04d07a0c83d2cc0c0a9b53cab9a6d, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:32,233 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965272233"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965272233"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965272233"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965272233"}]},"ts":"1675965272233"} 2023-02-09 17:54:32,233 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965270129 2023-02-09 17:54:32,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:32,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:32,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,234 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965270315 2023-02-09 17:54:32,235 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965270315 2023-02-09 17:54:32,236 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965270492 2023-02-09 17:54:32,236 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965270492 2023-02-09 17:54:32,236 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965270669 2023-02-09 17:54:32,236 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965270669 2023-02-09 17:54:32,237 INFO [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,237 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965270847 2023-02-09 17:54:32,238 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965270847 2023-02-09 17:54:32,239 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 2023-02-09 17:54:32,239 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 2023-02-09 17:54:32,239 INFO [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 58cef6fb1a2ed5ac5e50a58a3e6d9dea columnFamilyName cf1 2023-02-09 17:54:32,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-09 17:54:32,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 1ee04d07a0c83d2cc0c0a9b53cab9a6d, server=jenkins-hbase4.apache.org,44623,1675965244508 in 252 msec 2023-02-09 17:54:32,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, ASSIGN in 410 msec 2023-02-09 17:54:32,250 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f-top 2023-02-09 17:54:32,252 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] throttle.PressureAwareThroughputController(145): 1ee04d07a0c83d2cc0c0a9b53cab9a6d#cf2#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,253 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ee04d07a0c83d2cc0c0a9b53cab9a6d#cf1#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,260 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9-top 2023-02-09 17:54:32,263 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,263 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452-top 2023-02-09 17:54:32,266 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1-top 2023-02-09 17:54:32,268 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:32,268 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:32,272 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:32,272 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:32,276 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:32,276 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:32,276 INFO [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(310): Store=58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:32,276 INFO [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,277 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp/cf1/d1c58c5c3cb8479597273fe6f933b4a8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d1c58c5c3cb8479597273fe6f933b4a8 2023-02-09 17:54:32,277 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp/cf2/10c01bc3af5346afab426d75e85ff69a as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/10c01bc3af5346afab426d75e85ff69a 2023-02-09 17:54:32,277 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 2023-02-09 17:54:32,277 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 2023-02-09 17:54:32,278 INFO [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 58cef6fb1a2ed5ac5e50a58a3e6d9dea columnFamilyName cf2 2023-02-09 17:54:32,283 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 of 1ee04d07a0c83d2cc0c0a9b53cab9a6d into 10c01bc3af5346afab426d75e85ff69a(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,283 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1ee04d07a0c83d2cc0c0a9b53cab9a6d: 2023-02-09 17:54:32,283 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 of 1ee04d07a0c83d2cc0c0a9b53cab9a6d into d1c58c5c3cb8479597273fe6f933b4a8(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,283 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d., storeName=1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2, priority=9, startTime=1675965272230; duration=0sec 2023-02-09 17:54:32,283 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1ee04d07a0c83d2cc0c0a9b53cab9a6d: 2023-02-09 17:54:32,283 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d., storeName=1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1, priority=9, startTime=1675965272229; duration=0sec 2023-02-09 17:54:32,283 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,283 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,287 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41-top 2023-02-09 17:54:32,290 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,290 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c-top 2023-02-09 17:54:32,293 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661-top 2023-02-09 17:54:32,296 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4-top 2023-02-09 17:54:32,301 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:32,301 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:32,304 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:32,304 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:32,309 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:32,309 DEBUG [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:32,310 INFO [StoreOpener-58cef6fb1a2ed5ac5e50a58a3e6d9dea-1] regionserver.HStore(310): Store=58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:32,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:54:32,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 58cef6fb1a2ed5ac5e50a58a3e6d9dea; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70175507, jitterRate=0.04569654166698456}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:54:32,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 58cef6fb1a2ed5ac5e50a58a3e6d9dea: Running coprocessor pre-open hook at 1675965272234Writing region info on filesystem at 1675965272234Initializing all the Stores at 1675965272236 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965272236Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965272236Cleaning up temporary data from old regions at 1675965272312 (+76 ms)Cleaning up detritus from prior splits at 1675965272312Running coprocessor post-open hooks at 1675965272314 (+2 ms)Region opened successfully at 1675965272314 2023-02-09 17:54:32,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea., pid=78, masterSystemTime=1675965272138 2023-02-09 17:54:32,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:32,316 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:32,316 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:54:32,316 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,316 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1912): 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 is initiating minor compaction (all files) 2023-02-09 17:54:32,316 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,317 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,317 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1912): 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 is initiating minor compaction (all files) 2023-02-09 17:54:32,317 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,317 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp, totalSize=34.8 K 2023-02-09 17:54:32,317 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp, totalSize=34.8 K 2023-02-09 17:54:32,317 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965269770 2023-02-09 17:54:32,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,318 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965269770 2023-02-09 17:54:32,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,318 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965269943 2023-02-09 17:54:32,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=58cef6fb1a2ed5ac5e50a58a3e6d9dea, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:32,318 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965269943 2023-02-09 17:54:32,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965272318"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965272318"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965272318"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965272318"}]},"ts":"1675965272318"} 2023-02-09 17:54:32,319 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965270129 2023-02-09 17:54:32,319 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965270129 2023-02-09 17:54:32,319 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675965270847 2023-02-09 17:54:32,320 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675965270847 2023-02-09 17:54:32,320 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,320 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,320 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,321 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,321 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,321 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-09 17:54:32,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 58cef6fb1a2ed5ac5e50a58a3e6d9dea, server=jenkins-hbase4.apache.org,44623,1675965244508 in 330 msec 2023-02-09 17:54:32,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-09 17:54:32,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, ASSIGN in 493 msec 2023-02-09 17:54:32,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea in 917 msec 2023-02-09 17:54:32,340 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] throttle.PressureAwareThroughputController(145): 58cef6fb1a2ed5ac5e50a58a3e6d9dea#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,340 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] throttle.PressureAwareThroughputController(145): 58cef6fb1a2ed5ac5e50a58a3e6d9dea#cf1#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,360 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0200ee03a7b848539720fccb29750597: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,361 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 2fcc736a10304c7dbbd038ef851afd48: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,361 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp/cf2/0200ee03a7b848539720fccb29750597 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/0200ee03a7b848539720fccb29750597 2023-02-09 17:54:32,362 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp/cf1/2fcc736a10304c7dbbd038ef851afd48 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2fcc736a10304c7dbbd038ef851afd48 2023-02-09 17:54:32,365 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0200ee03a7b848539720fccb29750597: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,366 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 2fcc736a10304c7dbbd038ef851afd48: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,366 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 of 58cef6fb1a2ed5ac5e50a58a3e6d9dea into 0200ee03a7b848539720fccb29750597(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,367 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 58cef6fb1a2ed5ac5e50a58a3e6d9dea: 2023-02-09 17:54:32,367 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea., storeName=58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2, priority=9, startTime=1675965272316; duration=0sec 2023-02-09 17:54:32,367 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 of 58cef6fb1a2ed5ac5e50a58a3e6d9dea into 2fcc736a10304c7dbbd038ef851afd48(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,367 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,367 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 58cef6fb1a2ed5ac5e50a58a3e6d9dea: 2023-02-09 17:54:32,367 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea., storeName=58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1, priority=9, startTime=1675965272315; duration=0sec 2023-02-09 17:54:32,367 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,420 DEBUG [Listener at localhost/34581] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 17:54:32,423 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42812, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 17:54:32,424 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 17:54:32,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.HStore(1912): 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 is initiating major compaction (all files) 2023-02-09 17:54:32,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7a8011e6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,425 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 17:54:32,425 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d1c58c5c3cb8479597273fe6f933b4a8] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp, totalSize=6.8 K 2023-02-09 17:54:32,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.HStore(1912): 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 is initiating major compaction (all files) 2023-02-09 17:54:32,425 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@66458a69; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,425 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,425 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/10c01bc3af5346afab426d75e85ff69a] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp, totalSize=6.8 K 2023-02-09 17:54:32,425 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting d1c58c5c3cb8479597273fe6f933b4a8, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965269770 2023-02-09 17:54:32,426 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 10c01bc3af5346afab426d75e85ff69a, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965269770 2023-02-09 17:54:32,427 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 17:54:32,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.HStore(1912): 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 is initiating major compaction (all files) 2023-02-09 17:54:32,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@279c3c4a; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:54:32,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 17:54:32,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.HStore(1912): 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 is initiating major compaction (all files) 2023-02-09 17:54:32,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@187264ea; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 17:54:32,428 INFO [Listener at localhost/34581] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 17:54:32,430 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1ee04d07a0c83d2cc0c0a9b53cab9a6d#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,430 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] throttle.PressureAwareThroughputController(145): 1ee04d07a0c83d2cc0c0a9b53cab9a6d#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,438 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. is not mergeable because it has references 2023-02-09 17:54:32,438 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. is not mergeable because it has references 2023-02-09 17:54:32,441 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp/cf2/189dd69b93d6442bb25849e7ff51f1ba as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/189dd69b93d6442bb25849e7ff51f1ba 2023-02-09 17:54:32,442 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/.tmp/cf1/d16668d00db94478875a2687255c7871 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d16668d00db94478875a2687255c7871 2023-02-09 17:54:32,445 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2 of 1ee04d07a0c83d2cc0c0a9b53cab9a6d into 189dd69b93d6442bb25849e7ff51f1ba(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,445 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1ee04d07a0c83d2cc0c0a9b53cab9a6d: 2023-02-09 17:54:32,446 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1 of 1ee04d07a0c83d2cc0c0a9b53cab9a6d into d16668d00db94478875a2687255c7871(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,446 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1ee04d07a0c83d2cc0c0a9b53cab9a6d: 2023-02-09 17:54:32,447 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675965272425; duration=0sec 2023-02-09 17:54:32,447 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675965272425; duration=0sec 2023-02-09 17:54:32,447 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 17:54:32,447 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 17:54:32,447 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,447 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,447 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2fcc736a10304c7dbbd038ef851afd48] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp, totalSize=5.0 K 2023-02-09 17:54:32,447 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/0200ee03a7b848539720fccb29750597] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp, totalSize=5.0 K 2023-02-09 17:54:32,447 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] compactions.Compactor(207): Compacting 2fcc736a10304c7dbbd038ef851afd48, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,447 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] compactions.Compactor(207): Compacting 0200ee03a7b848539720fccb29750597, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 17:54:32,450 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] throttle.PressureAwareThroughputController(145): 58cef6fb1a2ed5ac5e50a58a3e6d9dea#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,450 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] throttle.PressureAwareThroughputController(145): 58cef6fb1a2ed5ac5e50a58a3e6d9dea#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:54:32,463 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 2bbc6d3ae2314a40a28245de05d67d10: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,464 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp/cf1/2bbc6d3ae2314a40a28245de05d67d10 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2bbc6d3ae2314a40a28245de05d67d10 2023-02-09 17:54:32,469 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 2bbc6d3ae2314a40a28245de05d67d10: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,469 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 01689bc8c3fa441191b1baa3b5183879: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,470 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/.tmp/cf2/01689bc8c3fa441191b1baa3b5183879 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/01689bc8c3fa441191b1baa3b5183879 2023-02-09 17:54:32,470 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1 of 58cef6fb1a2ed5ac5e50a58a3e6d9dea into 2bbc6d3ae2314a40a28245de05d67d10(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,470 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 58cef6fb1a2ed5ac5e50a58a3e6d9dea: 2023-02-09 17:54:32,470 INFO [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675965272427; duration=0sec 2023-02-09 17:54:32,470 DEBUG [RS:0;jenkins-hbase4:44623-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,472 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 01689bc8c3fa441191b1baa3b5183879: NONE, but ROW specified in column family configuration 2023-02-09 17:54:32,473 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2 of 58cef6fb1a2ed5ac5e50a58a3e6d9dea into 01689bc8c3fa441191b1baa3b5183879(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:54:32,473 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 58cef6fb1a2ed5ac5e50a58a3e6d9dea: 2023-02-09 17:54:32,473 INFO [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675965272427; duration=0sec 2023-02-09 17:54:32,474 DEBUG [RS:0;jenkins-hbase4:44623-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:54:32,546 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44623] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. is not mergeable because it has references 2023-02-09 17:54:32,547 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44623] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. is not mergeable because it has references 2023-02-09 17:54:32,553 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-09 17:54:32,555 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:32,555 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,556 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,557 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,557 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,557 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,557 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,560 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,560 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 17:54:32,561 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965272561"}]},"ts":"1675965272561"} 2023-02-09 17:54:32,562 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-09 17:54:32,563 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-09 17:54:32,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, UNASSIGN}] 2023-02-09 17:54:32,565 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, UNASSIGN 2023-02-09 17:54:32,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=58cef6fb1a2ed5ac5e50a58a3e6d9dea, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:32,566 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965272565"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965272565"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965272565"}]},"ts":"1675965272565"} 2023-02-09 17:54:32,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 58cef6fb1a2ed5ac5e50a58a3e6d9dea, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:32,569 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, UNASSIGN 2023-02-09 17:54:32,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=1ee04d07a0c83d2cc0c0a9b53cab9a6d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:54:32,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965272569"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965272569"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965272569"}]},"ts":"1675965272569"} 2023-02-09 17:54:32,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 1ee04d07a0c83d2cc0c0a9b53cab9a6d, server=jenkins-hbase4.apache.org,44623,1675965244508}] 2023-02-09 17:54:32,661 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,661 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 17:54:32,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 58cef6fb1a2ed5ac5e50a58a3e6d9dea, disabling compactions & flushes 2023-02-09 17:54:32,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. after waiting 0 ms 2023-02-09 17:54:32,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,719 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2fcc736a10304c7dbbd038ef851afd48, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47] to archive 2023-02-09 17:54:32,720 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:32,722 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,723 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,724 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,725 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,726 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:32,727 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:32,728 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2fcc736a10304c7dbbd038ef851afd48 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2fcc736a10304c7dbbd038ef851afd48 2023-02-09 17:54:32,729 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:32,731 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/0200ee03a7b848539720fccb29750597, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa] to archive 2023-02-09 17:54:32,731 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:32,733 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,734 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,735 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,736 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,737 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:32,738 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:32,738 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/0200ee03a7b848539720fccb29750597 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/0200ee03a7b848539720fccb29750597 2023-02-09 17:54:32,740 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:32,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 17:54:32,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:32,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea. 2023-02-09 17:54:32,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 58cef6fb1a2ed5ac5e50a58a3e6d9dea: Waiting for close lock at 1675965272719Running coprocessor pre-close hooks at 1675965272719Disabling compacts and flushes for region at 1675965272719Disabling writes for close at 1675965272719Writing region close event to WAL at 1675965272741 (+22 ms)Running coprocessor post-close hooks at 1675965272744 (+3 ms)Closed at 1675965272744 2023-02-09 17:54:32,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1ee04d07a0c83d2cc0c0a9b53cab9a6d, disabling compactions & flushes 2023-02-09 17:54:32,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. after waiting 0 ms 2023-02-09 17:54:32,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=58cef6fb1a2ed5ac5e50a58a3e6d9dea, regionState=CLOSED 2023-02-09 17:54:32,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965272746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965272746"}]},"ts":"1675965272746"} 2023-02-09 17:54:32,746 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d1c58c5c3cb8479597273fe6f933b4a8, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452-bottom] to archive 2023-02-09 17:54:32,747 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:32,749 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/a276e8a48aff4bd5b93e929a730cdaf9.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-09 17:54:32,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 58cef6fb1a2ed5ac5e50a58a3e6d9dea, server=jenkins-hbase4.apache.org,44623,1675965244508 in 181 msec 2023-02-09 17:54:32,750 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/f369eccc37a04602b8f82f215b9869e1.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=58cef6fb1a2ed5ac5e50a58a3e6d9dea, UNASSIGN in 185 msec 2023-02-09 17:54:32,751 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/2221f706f4eb48e084a375040d8c4b7f.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,752 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-2c022be3116c4fcc92e166efcebc76d7 2023-02-09 17:54:32,753 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-f3feaebeb633472db33a173c5bab0485 2023-02-09 17:54:32,754 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-b181a52658254698ab8c1c1a2467398d 2023-02-09 17:54:32,755 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d1c58c5c3cb8479597273fe6f933b4a8 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d1c58c5c3cb8479597273fe6f933b4a8 2023-02-09 17:54:32,756 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,758 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/10c01bc3af5346afab426d75e85ff69a, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c-bottom] to archive 2023-02-09 17:54:32,758 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:54:32,759 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/cd1ff917f47b4bc995e843e5271b7cc4.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,760 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/19858728acaf4b7ab41f657dcca07c41.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,761 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/62261d7adc0d4e068ea88563711be661.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,762 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c70fa46d0e514894b5134c9b6d1453c1 2023-02-09 17:54:32,763 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-6fd182adbf154d478d23434fdd9aae8b 2023-02-09 17:54:32,764 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/testSplitTableRegionDeletedRowsDaughter=4ef9f57187c5579008833c2bab1cfd4d-c16fa9a435bc49b598df6fe3d347d2cf 2023-02-09 17:54:32,765 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/10c01bc3af5346afab426d75e85ff69a to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/10c01bc3af5346afab426d75e85ff69a 2023-02-09 17:54:32,765 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/595488a797174f7fa3165864d2a4f06c.4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 17:54:32,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:32,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d. 2023-02-09 17:54:32,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1ee04d07a0c83d2cc0c0a9b53cab9a6d: Waiting for close lock at 1675965272745Running coprocessor pre-close hooks at 1675965272745Disabling compacts and flushes for region at 1675965272745Disabling writes for close at 1675965272745Writing region close event to WAL at 1675965272766 (+21 ms)Running coprocessor post-close hooks at 1675965272769 (+3 ms)Closed at 1675965272770 (+1 ms) 2023-02-09 17:54:32,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=1ee04d07a0c83d2cc0c0a9b53cab9a6d, regionState=CLOSED 2023-02-09 17:54:32,772 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675965272772"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965272772"}]},"ts":"1675965272772"} 2023-02-09 17:54:32,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 17:54:32,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 1ee04d07a0c83d2cc0c0a9b53cab9a6d, server=jenkins-hbase4.apache.org,44623,1675965244508 in 203 msec 2023-02-09 17:54:32,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-09 17:54:32,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1ee04d07a0c83d2cc0c0a9b53cab9a6d, UNASSIGN in 211 msec 2023-02-09 17:54:32,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965272777"}]},"ts":"1675965272777"} 2023-02-09 17:54:32,779 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-09 17:54:32,780 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-09 17:54:32,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 224 msec 2023-02-09 17:54:32,862 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,862 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 17:54:32,864 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-09 17:54:32,864 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,864 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,864 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,867 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,867 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,867 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 17:54:32,868 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,872 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,872 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,872 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,874 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/recovered.edits] 2023-02-09 17:54:32,874 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/recovered.edits] 2023-02-09 17:54:32,874 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/recovered.edits] 2023-02-09 17:54:32,877 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d16668d00db94478875a2687255c7871 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf1/d16668d00db94478875a2687255c7871 2023-02-09 17:54:32,877 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2221f706f4eb48e084a375040d8c4b7f 2023-02-09 17:54:32,877 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2bbc6d3ae2314a40a28245de05d67d10 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf1/2bbc6d3ae2314a40a28245de05d67d10 2023-02-09 17:54:32,879 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/2c022be3116c4fcc92e166efcebc76d7 2023-02-09 17:54:32,880 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/189dd69b93d6442bb25849e7ff51f1ba to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/cf2/189dd69b93d6442bb25849e7ff51f1ba 2023-02-09 17:54:32,880 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/7e0ee8e1df3e4905a54e4e9aa40a2b47 2023-02-09 17:54:32,880 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/01689bc8c3fa441191b1baa3b5183879 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/cf2/01689bc8c3fa441191b1baa3b5183879 2023-02-09 17:54:32,882 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/a276e8a48aff4bd5b93e929a730cdaf9 2023-02-09 17:54:32,883 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/b181a52658254698ab8c1c1a2467398d 2023-02-09 17:54:32,883 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/recovered.edits/113.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d/recovered.edits/113.seqid 2023-02-09 17:54:32,884 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/recovered.edits/113.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea/recovered.edits/113.seqid 2023-02-09 17:54:32,884 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1ee04d07a0c83d2cc0c0a9b53cab9a6d 2023-02-09 17:54:32,884 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:54:32,885 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/de610e3c8e0f42ce8e1e0d1c0e820452 2023-02-09 17:54:32,886 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/e547245163834ea5bc9b614630ff9903 2023-02-09 17:54:32,887 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f369eccc37a04602b8f82f215b9869e1 2023-02-09 17:54:32,888 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/f3feaebeb633472db33a173c5bab0485 2023-02-09 17:54:32,889 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf1/ffb7f0ce8f1f4f33a678df44b4d813b5 2023-02-09 17:54:32,892 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/19858728acaf4b7ab41f657dcca07c41 2023-02-09 17:54:32,893 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/4a0eb617ed7542cbb9e8af7860c6b57b 2023-02-09 17:54:32,894 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/595488a797174f7fa3165864d2a4f06c 2023-02-09 17:54:32,895 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/62261d7adc0d4e068ea88563711be661 2023-02-09 17:54:32,896 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/6fd182adbf154d478d23434fdd9aae8b 2023-02-09 17:54:32,897 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c16fa9a435bc49b598df6fe3d347d2cf 2023-02-09 17:54:32,899 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/c70fa46d0e514894b5134c9b6d1453c1 2023-02-09 17:54:32,900 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/cd1ff917f47b4bc995e843e5271b7cc4 2023-02-09 17:54:32,901 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/d301baa024b24ff29a650b619aafeec0 2023-02-09 17:54:32,902 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/cf2/e17e66d70e4d4e58921731d22a6adbfa 2023-02-09 17:54:32,904 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/recovered.edits/106.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d/recovered.edits/106.seqid 2023-02-09 17:54:32,904 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4ef9f57187c5579008833c2bab1cfd4d 2023-02-09 17:54:32,904 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 17:54:32,906 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,908 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-09 17:54:32,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-09 17:54:32,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-09 17:54:32,911 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965272910"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:32,911 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965272910"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:32,911 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965272910"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:32,912 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:54:32,912 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4ef9f57187c5579008833c2bab1cfd4d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675965269266.4ef9f57187c5579008833c2bab1cfd4d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 1ee04d07a0c83d2cc0c0a9b53cab9a6d, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675965271409.1ee04d07a0c83d2cc0c0a9b53cab9a6d.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 58cef6fb1a2ed5ac5e50a58a3e6d9dea, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675965271409.58cef6fb1a2ed5ac5e50a58a3e6d9dea.', STARTKEY => '60', ENDKEY => ''}] 2023-02-09 17:54:32,912 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-09 17:54:32,912 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965272912"}]},"ts":"9223372036854775807"} 2023-02-09 17:54:32,913 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-09 17:54:32,916 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:54:32,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-02-09 17:54:32,968 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,968 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:32,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 17:54:32,969 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-09 17:54:32,984 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 407) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:45100 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:44800 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:55158 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:55246 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36702 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:35652 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36936 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:54612 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:45174 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:55220 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:44888 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:42218 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:42332 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6d6b0340-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:36492 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:35608 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1817084602_15 at /127.0.0.1:55174 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 853) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=340 (was 340), ProcessCount=168 (was 168), AvailableMemoryMB=3213 (was 3283) 2023-02-09 17:54:32,998 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=411, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=168, AvailableMemoryMB=3213 2023-02-09 17:54:32,998 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:54:32,998 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:32,998 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:33,001 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:54:33,001 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:54:33,002 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:54:33,004 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:54:33,004 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:54:33,005 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,006 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 empty. 2023-02-09 17:54:33,006 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,006 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 17:54:33,015 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:33,017 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 520e862302f7f9f2f3a223d4e5ede955, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:54:33,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:33,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 520e862302f7f9f2f3a223d4e5ede955, disabling compactions & flushes 2023-02-09 17:54:33,023 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. after waiting 0 ms 2023-02-09 17:54:33,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,023 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,023 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 520e862302f7f9f2f3a223d4e5ede955: Waiting for close lock at 1675965273023Disabling compacts and flushes for region at 1675965273023Disabling writes for close at 1675965273023Writing region close event to WAL at 1675965273023Closed at 1675965273023 2023-02-09 17:54:33,025 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:54:33,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965273025"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965273025"}]},"ts":"1675965273025"} 2023-02-09 17:54:33,027 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:54:33,027 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:54:33,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965273027"}]},"ts":"1675965273027"} 2023-02-09 17:54:33,028 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-09 17:54:33,031 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:33,031 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:33,031 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:33,031 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:33,031 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:54:33,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, ASSIGN}] 2023-02-09 17:54:33,033 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, ASSIGN 2023-02-09 17:54:33,033 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:54:33,184 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:33,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:33,187 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965273186"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965273186"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965273186"}]},"ts":"1675965273186"} 2023-02-09 17:54:33,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:33,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 520e862302f7f9f2f3a223d4e5ede955, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:33,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:33,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:33,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,345 INFO [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,346 DEBUG [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1 2023-02-09 17:54:33,346 DEBUG [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1 2023-02-09 17:54:33,347 INFO [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 520e862302f7f9f2f3a223d4e5ede955 columnFamilyName cf1 2023-02-09 17:54:33,347 INFO [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] regionserver.HStore(310): Store=520e862302f7f9f2f3a223d4e5ede955/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:33,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:54:33,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 520e862302f7f9f2f3a223d4e5ede955; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67363521, jitterRate=0.0037946850061416626}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 17:54:33,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 520e862302f7f9f2f3a223d4e5ede955: Running coprocessor pre-open hook at 1675965273344Writing region info on filesystem at 1675965273344Initializing all the Stores at 1675965273345 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965273345Cleaning up temporary data from old regions at 1675965273357 (+12 ms)Cleaning up detritus from prior splits at 1675965273357Running coprocessor post-open hooks at 1675965273361 (+4 ms)Region opened successfully at 1675965273361 2023-02-09 17:54:33,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955., pid=87, masterSystemTime=1675965273340 2023-02-09 17:54:33,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,363 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:33,364 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965273363"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965273363"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965273363"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965273363"}]},"ts":"1675965273363"} 2023-02-09 17:54:33,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-09 17:54:33,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552 in 177 msec 2023-02-09 17:54:33,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-09 17:54:33,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, ASSIGN in 334 msec 2023-02-09 17:54:33,368 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:54:33,368 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965273368"}]},"ts":"1675965273368"} 2023-02-09 17:54:33,369 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-09 17:54:33,374 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:54:33,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 373 msec 2023-02-09 17:54:33,503 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:33,503 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:33,505 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:54:33,505 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:54:33,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 17:54:33,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:54:33,531 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 17:54:33,532 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 17:54:33,532 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 17:54:33,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-09 17:54:33,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, REOPEN/MOVE}] 2023-02-09 17:54:33,547 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, REOPEN/MOVE 2023-02-09 17:54:33,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:33,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965273547"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965273547"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965273547"}]},"ts":"1675965273547"} 2023-02-09 17:54:33,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:33,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 520e862302f7f9f2f3a223d4e5ede955, disabling compactions & flushes 2023-02-09 17:54:33,702 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. after waiting 0 ms 2023-02-09 17:54:33,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 17:54:33,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:54:33,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:33,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 520e862302f7f9f2f3a223d4e5ede955: Waiting for close lock at 1675965273701Running coprocessor pre-close hooks at 1675965273701Disabling compacts and flushes for region at 1675965273701Disabling writes for close at 1675965273702 (+1 ms)Writing region close event to WAL at 1675965273702Running coprocessor post-close hooks at 1675965273705 (+3 ms)Closed at 1675965273705 2023-02-09 17:54:33,705 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 520e862302f7f9f2f3a223d4e5ede955 to self. 2023-02-09 17:54:33,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:33,707 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=CLOSED 2023-02-09 17:54:33,707 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965273707"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965273707"}]},"ts":"1675965273707"} 2023-02-09 17:54:33,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-09 17:54:33,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552 in 159 msec 2023-02-09 17:54:33,710 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=true 2023-02-09 17:54:33,860 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:54:33,860 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:33,861 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965273860"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965273860"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965273860"}]},"ts":"1675965273860"} 2023-02-09 17:54:33,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:54:34,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 520e862302f7f9f2f3a223d4e5ede955, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:54:34,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,018 INFO [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,019 DEBUG [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1 2023-02-09 17:54:34,019 DEBUG [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1 2023-02-09 17:54:34,020 INFO [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 520e862302f7f9f2f3a223d4e5ede955 columnFamilyName cf1 2023-02-09 17:54:34,020 INFO [StoreOpener-520e862302f7f9f2f3a223d4e5ede955-1] regionserver.HStore(310): Store=520e862302f7f9f2f3a223d4e5ede955/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:54:34,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:54:34,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 520e862302f7f9f2f3a223d4e5ede955; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61844351, jitterRate=-0.0784473568201065}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 17:54:34,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 520e862302f7f9f2f3a223d4e5ede955: Running coprocessor pre-open hook at 1675965274017Writing region info on filesystem at 1675965274017Initializing all the Stores at 1675965274018 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965274018Cleaning up temporary data from old regions at 1675965274022 (+4 ms)Cleaning up detritus from prior splits at 1675965274022Running coprocessor post-open hooks at 1675965274024 (+2 ms)Region opened successfully at 1675965274024 2023-02-09 17:54:34,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955., pid=92, masterSystemTime=1675965274014 2023-02-09 17:54:34,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:34,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:54:34,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:54:34,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965274027"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965274027"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965274027"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965274027"}]},"ts":"1675965274027"} 2023-02-09 17:54:34,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-09 17:54:34,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552 in 166 msec 2023-02-09 17:54:34,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-09 17:54:34,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, REOPEN/MOVE in 483 msec 2023-02-09 17:54:34,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-09 17:54:34,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 498 msec 2023-02-09 17:54:34,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, ASSIGN}] 2023-02-09 17:54:34,037 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, ASSIGN 2023-02-09 17:54:34,038 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 17:54:34,188 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:54:34,189 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:54:34,189 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:54:34,189 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:54:34,189 DEBUG [jenkins-hbase4:43955] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-09 17:54:34,190 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=0389f4cb546df1d55fd1fdc67418f228, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:54:34,190 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965274190"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675965274190"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675965274190"}]},"ts":"1675965274190"} 2023-02-09 17:54:34,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 0389f4cb546df1d55fd1fdc67418f228, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:54:34,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:54:34,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0389f4cb546df1d55fd1fdc67418f228, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-09 17:54:34,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:54:34,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0389f4cb546df1d55fd1fdc67418f228 2023-02-09 17:54:34,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:54:34,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0389f4cb546df1d55fd1fdc67418f228 2023-02-09 17:54:34,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0389f4cb546df1d55fd1fdc67418f228 2023-02-09 17:54:34,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:34,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:34,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:34,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:34,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:34,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:34,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:35,910 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-09 17:54:35,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:36,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,107 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:54:37,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:37,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:38,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:39,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:40,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:41,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:42,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:43,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:44,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:45,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:46,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:47,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:48,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:49,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:50,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:51,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:52,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:53,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:54,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:55,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:56,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:57,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:58,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:54:59,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:00,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:01,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:02,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:03,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:04,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,909 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 17:55:05,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:05,915 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 17:55:05,917 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 17:55:06,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:06,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:07,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:08,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:09,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:10,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:11,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:12,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:13,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:14,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:15,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:16,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:17,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:18,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:19,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:20,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:21,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:22,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:23,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:24,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:25,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:26,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:27,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:28,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:29,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:30,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:31,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:32,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,627 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:33,627 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:33,630 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:33,631 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:33,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 17:55:33,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,731 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:33,731 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:33,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 17:55:33,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:33,932 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:33,932 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:33,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 17:55:33,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,233 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:34,233 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:34,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 17:55:34,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,734 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:34,734 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:34,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 17:55:34,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:34,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,737 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:35,738 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:35,739 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:35,739 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:35,739 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:35,740 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 520e862302f7f9f2f3a223d4e5ede955, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:35,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 exec-time=5 msec 2023-02-09 17:55:35,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:35,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 17:55:36,076 INFO [StoreOpener-0389f4cb546df1d55fd1fdc67418f228-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0389f4cb546df1d55fd1fdc67418f228 2023-02-09 17:55:36,080 DEBUG [StoreOpener-0389f4cb546df1d55fd1fdc67418f228-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1 2023-02-09 17:55:36,080 DEBUG [StoreOpener-0389f4cb546df1d55fd1fdc67418f228-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1 2023-02-09 17:55:36,081 INFO [StoreOpener-0389f4cb546df1d55fd1fdc67418f228-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0389f4cb546df1d55fd1fdc67418f228 columnFamilyName cf1 2023-02-09 17:55:36,082 INFO [StoreOpener-0389f4cb546df1d55fd1fdc67418f228-1] regionserver.HStore(310): Store=0389f4cb546df1d55fd1fdc67418f228/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:36,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0389f4cb546df1d55fd1fdc67418f228; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68638047, jitterRate=0.02278660237789154}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 17:55:36,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0389f4cb546df1d55fd1fdc67418f228: Running coprocessor pre-open hook at 1675965274352Initializing all the Stores at 1675965336075 (+61723 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965336076 (+1 ms)Running coprocessor post-open hooks at 1675965336083 (+7 ms)Region opened successfully at 1675965336083 2023-02-09 17:55:36,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228., pid=94, masterSystemTime=1675965274345 2023-02-09 17:55:36,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:55:36,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:55:36,085 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=0389f4cb546df1d55fd1fdc67418f228, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:36,085 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965336085"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1675965336085"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675965336085"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675965336085"}]},"ts":"1675965336085"} 2023-02-09 17:55:36,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-09 17:55:36,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 0389f4cb546df1d55fd1fdc67418f228, server=jenkins-hbase4.apache.org,43923,1675965244585 in 1 mins, 1.895 sec 2023-02-09 17:55:36,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-09 17:55:36,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, ASSIGN in 1 mins, 2.052 sec 2023-02-09 17:55:36,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.575 sec 2023-02-09 17:55:36,092 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675965273509","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62582,"client":"172.31.14.131:48236","queuetimems":0,"class":"HMaster"} 2023-02-09 17:55:36,092 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-09 17:55:37,989 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up 2023-02-09 17:55:37,990 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:37,990 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:37,990 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:37,992 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:37,992 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:37,992 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:37,992 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:37,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:37,995 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:37,995 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:37,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 17:55:37,995 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965337995"}]},"ts":"1675965337995"} 2023-02-09 17:55:37,997 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-09 17:55:37,999 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-09 17:55:37,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, UNASSIGN}] 2023-02-09 17:55:38,000 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, UNASSIGN 2023-02-09 17:55:38,000 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=0389f4cb546df1d55fd1fdc67418f228, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:38,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965338000"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675965338000"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675965338000"}]},"ts":"1675965338000"} 2023-02-09 17:55:38,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 0389f4cb546df1d55fd1fdc67418f228, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:38,003 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, UNASSIGN 2023-02-09 17:55:38,004 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,004 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965338004"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965338004"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965338004"}]},"ts":"1675965338004"} 2023-02-09 17:55:38,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:38,096 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,096 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 17:55:38,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0389f4cb546df1d55fd1fdc67418f228 2023-02-09 17:55:38,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0389f4cb546df1d55fd1fdc67418f228, disabling compactions & flushes 2023-02-09 17:55:38,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:55:38,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:55:38,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. after waiting 0 ms 2023-02-09 17:55:38,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:55:38,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:38,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228. 2023-02-09 17:55:38,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0389f4cb546df1d55fd1fdc67418f228: Waiting for close lock at 1675965338153Running coprocessor pre-close hooks at 1675965338153Disabling compacts and flushes for region at 1675965338153Disabling writes for close at 1675965338154 (+1 ms)Writing region close event to WAL at 1675965338155 (+1 ms)Running coprocessor post-close hooks at 1675965338155Closed at 1675965338155 2023-02-09 17:55:38,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:55:38,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 520e862302f7f9f2f3a223d4e5ede955, disabling compactions & flushes 2023-02-09 17:55:38,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:55:38,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:55:38,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. after waiting 0 ms 2023-02-09 17:55:38,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:55:38,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0389f4cb546df1d55fd1fdc67418f228 2023-02-09 17:55:38,158 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=0389f4cb546df1d55fd1fdc67418f228, regionState=CLOSED 2023-02-09 17:55:38,158 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965338158"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675965338158"}]},"ts":"1675965338158"} 2023-02-09 17:55:38,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-09 17:55:38,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 0389f4cb546df1d55fd1fdc67418f228, server=jenkins-hbase4.apache.org,43923,1675965244585 in 158 msec 2023-02-09 17:55:38,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 17:55:38,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:38,163 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955. 2023-02-09 17:55:38,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 520e862302f7f9f2f3a223d4e5ede955: Waiting for close lock at 1675965338157Running coprocessor pre-close hooks at 1675965338157Disabling compacts and flushes for region at 1675965338157Disabling writes for close at 1675965338157Writing region close event to WAL at 1675965338158 (+1 ms)Running coprocessor post-close hooks at 1675965338162 (+4 ms)Closed at 1675965338163 (+1 ms) 2023-02-09 17:55:38,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228, UNASSIGN in 163 msec 2023-02-09 17:55:38,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:55:38,164 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=520e862302f7f9f2f3a223d4e5ede955, regionState=CLOSED 2023-02-09 17:55:38,164 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675965338164"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965338164"}]},"ts":"1675965338164"} 2023-02-09 17:55:38,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-09 17:55:38,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 520e862302f7f9f2f3a223d4e5ede955, server=jenkins-hbase4.apache.org,35803,1675965244552 in 160 msec 2023-02-09 17:55:38,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-09 17:55:38,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=520e862302f7f9f2f3a223d4e5ede955, UNASSIGN in 168 msec 2023-02-09 17:55:38,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965338168"}]},"ts":"1675965338168"} 2023-02-09 17:55:38,169 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-09 17:55:38,171 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-09 17:55:38,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 179 msec 2023-02-09 17:55:38,297 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,297 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 17:55:38,298 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-09 17:55:38,298 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,298 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,301 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,301 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,301 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,302 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 17:55:38,307 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:55:38,308 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/recovered.edits] 2023-02-09 17:55:38,312 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/recovered.edits/7.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955/recovered.edits/7.seqid 2023-02-09 17:55:38,313 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/520e862302f7f9f2f3a223d4e5ede955 2023-02-09 17:55:38,313 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 17:55:38,315 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,317 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-09 17:55:38,319 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-09 17:55:38,320 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,320 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-09 17:55:38,320 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965338320"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:38,320 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965338320"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:38,324 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-09 17:55:38,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 520e862302f7f9f2f3a223d4e5ede955, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675965273002.520e862302f7f9f2f3a223d4e5ede955.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 0389f4cb546df1d55fd1fdc67418f228, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675965273002_0001.0389f4cb546df1d55fd1fdc67418f228.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-09 17:55:38,324 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-09 17:55:38,325 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965338325"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:38,326 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-09 17:55:38,328 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:38,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 29 msec 2023-02-09 17:55:38,403 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,403 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 17:55:38,404 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-09 17:55:38,419 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 411), OpenFileDescriptor=653 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 340), ProcessCount=168 (was 168), AvailableMemoryMB=3084 (was 3213) 2023-02-09 17:55:38,431 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=653, MaxFileDescriptor=60000, SystemLoadAverage=183, ProcessCount=168, AvailableMemoryMB=3084 2023-02-09 17:55:38,432 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:38,432 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,432 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,434 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:55:38,435 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:55:38,436 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:38,437 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:55:38,438 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:55:38,439 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,439 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 empty. 2023-02-09 17:55:38,440 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,440 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 17:55:38,452 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-09 17:55:38,453 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2a3951a75479071f9743eaa175ca9ac6, NAME => 'testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:55:38,460 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:38,460 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 2a3951a75479071f9743eaa175ca9ac6, disabling compactions & flushes 2023-02-09 17:55:38,460 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,460 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,460 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. after waiting 0 ms 2023-02-09 17:55:38,460 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,460 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,460 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 2a3951a75479071f9743eaa175ca9ac6: Waiting for close lock at 1675965338460Disabling compacts and flushes for region at 1675965338460Disabling writes for close at 1675965338460Writing region close event to WAL at 1675965338460Closed at 1675965338460 2023-02-09 17:55:38,462 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:55:38,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675965338462"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965338462"}]},"ts":"1675965338462"} 2023-02-09 17:55:38,464 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:55:38,464 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:55:38,465 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965338465"}]},"ts":"1675965338465"} 2023-02-09 17:55:38,466 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-09 17:55:38,469 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:55:38,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:55:38,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:55:38,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:55:38,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:55:38,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, ASSIGN}] 2023-02-09 17:55:38,471 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, ASSIGN 2023-02-09 17:55:38,472 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:55:38,622 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:55:38,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=2a3951a75479071f9743eaa175ca9ac6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675965338623"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965338623"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965338623"}]},"ts":"1675965338623"} 2023-02-09 17:55:38,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 2a3951a75479071f9743eaa175ca9ac6, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:38,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2a3951a75479071f9743eaa175ca9ac6, NAME => 'testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:55:38,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:38,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:38,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,782 INFO [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,783 DEBUG [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1 2023-02-09 17:55:38,783 DEBUG [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1 2023-02-09 17:55:38,784 INFO [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a3951a75479071f9743eaa175ca9ac6 columnFamilyName cf1 2023-02-09 17:55:38,784 INFO [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] regionserver.HStore(310): Store=2a3951a75479071f9743eaa175ca9ac6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:38,784 INFO [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,785 DEBUG [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2 2023-02-09 17:55:38,786 DEBUG [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2 2023-02-09 17:55:38,786 INFO [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a3951a75479071f9743eaa175ca9ac6 columnFamilyName cf2 2023-02-09 17:55:38,786 INFO [StoreOpener-2a3951a75479071f9743eaa175ca9ac6-1] regionserver.HStore(310): Store=2a3951a75479071f9743eaa175ca9ac6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:38,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:38,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:38,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:55:38,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2a3951a75479071f9743eaa175ca9ac6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73726216, jitterRate=0.09860622882843018}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:38,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2a3951a75479071f9743eaa175ca9ac6: Running coprocessor pre-open hook at 1675965338781Writing region info on filesystem at 1675965338781Initializing all the Stores at 1675965338782 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965338782Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965338782Cleaning up temporary data from old regions at 1675965338788 (+6 ms)Cleaning up detritus from prior splits at 1675965338789 (+1 ms)Running coprocessor post-open hooks at 1675965338793 (+4 ms)Region opened successfully at 1675965338793 2023-02-09 17:55:38,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6., pid=104, masterSystemTime=1675965338777 2023-02-09 17:55:38,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,795 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=2a3951a75479071f9743eaa175ca9ac6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,795 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675965338795"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965338795"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965338795"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965338795"}]},"ts":"1675965338795"} 2023-02-09 17:55:38,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-09 17:55:38,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 2a3951a75479071f9743eaa175ca9ac6, server=jenkins-hbase4.apache.org,35803,1675965244552 in 172 msec 2023-02-09 17:55:38,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-09 17:55:38,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, ASSIGN in 328 msec 2023-02-09 17:55:38,800 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:55:38,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965338800"}]},"ts":"1675965338800"} 2023-02-09 17:55:38,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-09 17:55:38,803 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:55:38,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 368 msec 2023-02-09 17:55:38,943 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:38,943 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:38,944 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:38,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 17:55:38,945 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 17:55:38,945 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:38,945 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 17:55:38,945 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 17:55:38,946 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:38,946 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,947 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,947 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:38,947 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:38,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:38,947 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:38,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:38,947 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,947 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:38,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:38,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:38,948 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:38,948 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:38,948 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:38,948 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:38,949 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:38,949 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. started... 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:38,949 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:38,950 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:55:38,952 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:38,953 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:38,953 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:38,953 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:38,953 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:38,953 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:38,966 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/6771b5fabba440348655229f86a94d7d 2023-02-09 17:55:38,976 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/a557be1645224bc7b7f8f83103c82c1f 2023-02-09 17:55:38,980 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/6771b5fabba440348655229f86a94d7d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6771b5fabba440348655229f86a94d7d 2023-02-09 17:55:38,983 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6771b5fabba440348655229f86a94d7d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:38,984 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/a557be1645224bc7b7f8f83103c82c1f as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/a557be1645224bc7b7f8f83103c82c1f 2023-02-09 17:55:38,987 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/a557be1645224bc7b7f8f83103c82c1f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:38,987 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 37ms, sequenceid=6, compaction requested=false 2023-02-09 17:55:38,988 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-09 17:55:38,988 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:38,988 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:38,988 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:38,988 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:38,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:38,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:38,990 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:38,990 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:38,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:38,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:38,991 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:38,991 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:38,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:38,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:38,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 17:55:39,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f342942[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:39,006 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 17:55:39,006 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:39,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,007 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:39,008 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:39,008 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,008 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,010 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,010 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:39,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 17:55:39,013 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 17:55:39,013 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:39,013 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:39,013 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 17:55:39,013 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,015 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,016 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,016 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,018 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,018 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,018 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,021 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 17:55:39,022 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:39,021 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,022 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,022 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,022 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:39,022 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,023 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,123 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:39,123 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,123 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:39,139 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,139 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,139 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:39,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:39,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 17:55:39,145 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 17:55:39,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 17:55:39,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 17:55:39,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,145 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,147 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,147 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,147 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,147 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,147 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,147 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:39,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,149 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,149 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:39,149 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,150 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,150 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,150 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. started... 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,150 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,150 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,156 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,157 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,157 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,157 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,158 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,160 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/47e2da4b3001473db24a569a64504200 2023-02-09 17:55:39,177 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/9f5ddde55832491fad18894191f9361e 2023-02-09 17:55:39,185 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/47e2da4b3001473db24a569a64504200 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/47e2da4b3001473db24a569a64504200 2023-02-09 17:55:39,190 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/47e2da4b3001473db24a569a64504200, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:39,192 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/9f5ddde55832491fad18894191f9361e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/9f5ddde55832491fad18894191f9361e 2023-02-09 17:55:39,197 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/9f5ddde55832491fad18894191f9361e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:39,198 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 48ms, sequenceid=19, compaction requested=false 2023-02-09 17:55:39,198 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:39,198 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,199 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,200 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,201 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,202 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,202 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 17:55:39,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cf06859[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:39,204 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 17:55:39,204 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,205 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,205 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,206 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,206 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,206 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,206 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,210 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,210 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,210 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,209 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,210 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,210 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:39,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 17:55:39,214 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 17:55:39,214 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:39,214 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:39,214 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 17:55:39,214 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:39,215 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,215 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,216 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,215 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,217 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,217 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,218 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,219 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,219 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,220 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,222 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,222 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,222 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 17:55:39,223 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:39,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,223 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,323 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:39,323 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,324 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:39,335 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,335 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,336 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:39,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:39,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 17:55:39,337 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 17:55:39,337 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,337 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 17:55:39,337 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 17:55:39,338 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,338 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,339 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,339 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,339 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,339 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,339 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,339 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:39,341 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,341 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:39,341 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. started... 2023-02-09 17:55:39,342 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:39,343 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,344 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,344 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,344 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,344 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,344 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,356 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/6af20766c4984fe38dddd4e1a26aa8ba 2023-02-09 17:55:39,367 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/d2c076031487403e849f844fd6d20b0a 2023-02-09 17:55:39,371 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/6af20766c4984fe38dddd4e1a26aa8ba as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6af20766c4984fe38dddd4e1a26aa8ba 2023-02-09 17:55:39,375 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6af20766c4984fe38dddd4e1a26aa8ba, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:39,375 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/d2c076031487403e849f844fd6d20b0a as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/d2c076031487403e849f844fd6d20b0a 2023-02-09 17:55:39,379 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/d2c076031487403e849f844fd6d20b0a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:39,380 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 39ms, sequenceid=32, compaction requested=true 2023-02-09 17:55:39,380 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:39,380 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:39,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:39,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,382 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,382 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,382 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,382 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 17:55:39,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a913564[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:39,384 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 17:55:39,384 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,385 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,385 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:39,389 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,389 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,389 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,390 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,390 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,390 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,390 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,391 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,391 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,391 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:39,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 17:55:39,393 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 17:55:39,394 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:39,394 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:39,394 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 17:55:39,394 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,395 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,396 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,396 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,398 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,398 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,398 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,405 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 17:55:39,405 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,405 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,405 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:39,406 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,406 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,406 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,406 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,406 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,406 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,506 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:39,506 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,506 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:39,518 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,519 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,519 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:39,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:39,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 17:55:39,521 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 17:55:39,521 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,521 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 17:55:39,521 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 17:55:39,521 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,521 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,522 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,522 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,522 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,522 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:39,524 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,524 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:39,525 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. started... 2023-02-09 17:55:39,525 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:39,526 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,526 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,526 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,526 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,526 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,526 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,533 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/afbaa9ba7849474a89c66d86c07d55c8 2023-02-09 17:55:39,543 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/96bf4b8a6c864bd4ba51caf4795ab8b7 2023-02-09 17:55:39,547 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/afbaa9ba7849474a89c66d86c07d55c8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/afbaa9ba7849474a89c66d86c07d55c8 2023-02-09 17:55:39,551 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/afbaa9ba7849474a89c66d86c07d55c8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:39,552 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/96bf4b8a6c864bd4ba51caf4795ab8b7 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/96bf4b8a6c864bd4ba51caf4795ab8b7 2023-02-09 17:55:39,557 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/96bf4b8a6c864bd4ba51caf4795ab8b7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:39,558 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 33ms, sequenceid=45, compaction requested=true 2023-02-09 17:55:39,558 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:39,558 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,558 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:39,558 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:39,558 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,558 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,558 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,561 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,561 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,561 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,561 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 17:55:39,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@556d2ecd[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:39,563 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 17:55:39,563 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,564 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:39,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,564 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:39,565 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:39,566 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,566 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,566 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,567 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,567 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,567 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,567 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,567 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,567 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,567 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:39,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 17:55:39,570 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 17:55:39,570 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:39,570 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:39,570 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 17:55:39,570 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,573 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,574 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,576 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,576 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,577 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,578 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,578 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,580 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,580 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 17:55:39,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,580 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:39,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,581 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,581 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,581 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:39,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,581 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,581 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:39,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,682 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:39,682 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,682 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:39,693 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,693 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,694 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:39,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:39,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 17:55:39,695 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 17:55:39,695 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,695 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 17:55:39,695 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 17:55:39,695 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,695 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,696 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,696 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,697 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,696 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,697 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,697 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:39,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,698 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,698 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,698 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,699 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,699 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,699 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. started... 2023-02-09 17:55:39,699 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:39,700 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,700 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,701 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,701 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,701 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,701 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,710 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/78df2e682a964413b6048e796f0b0b9e 2023-02-09 17:55:39,723 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/50412757b6de4b1ba520da9e8eefce6d 2023-02-09 17:55:39,727 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/78df2e682a964413b6048e796f0b0b9e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/78df2e682a964413b6048e796f0b0b9e 2023-02-09 17:55:39,730 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/78df2e682a964413b6048e796f0b0b9e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:39,731 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/50412757b6de4b1ba520da9e8eefce6d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/50412757b6de4b1ba520da9e8eefce6d 2023-02-09 17:55:39,735 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/50412757b6de4b1ba520da9e8eefce6d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:39,736 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 37ms, sequenceid=58, compaction requested=true 2023-02-09 17:55:39,736 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:39,736 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,736 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:39,736 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:39,736 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,736 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,736 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,739 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,739 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,739 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,739 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 17:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b8d731b[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:39,741 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 17:55:39,741 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,742 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:39,742 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,742 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,742 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:39,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,743 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,743 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,743 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:39,743 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:39,747 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,747 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,747 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,747 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,751 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,751 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,751 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,751 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,751 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,751 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:39,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 17:55:39,754 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 17:55:39,754 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:39,754 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:39,754 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 17:55:39,754 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:39,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,756 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,756 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,756 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,757 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,758 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,759 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,759 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,761 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 17:55:39,762 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:39,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,762 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:39,762 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,762 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,762 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,762 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:39,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,762 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,763 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,763 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,763 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,763 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,763 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,862 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:39,863 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,863 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:39,874 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:39,874 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:39,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:39,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:39,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 17:55:39,876 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 17:55:39,876 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,876 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 17:55:39,876 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 17:55:39,876 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,876 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,878 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,878 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,878 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,878 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,879 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,879 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 17:55:39,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 17:55:39,881 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:39,881 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:39,880 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,881 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,881 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,881 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. started... 2023-02-09 17:55:39,881 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 17:55:39,881 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,881 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,881 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:39,882 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,882 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,882 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,882 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,882 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,883 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,891 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/7c2f3037e677487f89a714158c17d1c8 2023-02-09 17:55:39,910 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/32718f1ee8f4401bbba70ca0b0e60eb0 2023-02-09 17:55:39,914 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/7c2f3037e677487f89a714158c17d1c8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/7c2f3037e677487f89a714158c17d1c8 2023-02-09 17:55:39,917 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/7c2f3037e677487f89a714158c17d1c8, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:39,918 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/32718f1ee8f4401bbba70ca0b0e60eb0 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/32718f1ee8f4401bbba70ca0b0e60eb0 2023-02-09 17:55:39,921 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/32718f1ee8f4401bbba70ca0b0e60eb0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:39,922 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 40ms, sequenceid=71, compaction requested=true 2023-02-09 17:55:39,922 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:39,922 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:39,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:39,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:39,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 17:55:39,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 17:55:39,923 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,923 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,924 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,924 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 17:55:39,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14606893[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:39,925 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 17:55:39,925 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,926 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,926 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,926 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:39,927 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,927 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 17:55:39,927 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:39,929 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,929 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,929 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,929 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,929 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,929 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,930 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 17:55:39,930 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,930 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,930 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 17:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:39,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 17:55:39,933 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 17:55:39,933 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:39,933 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:39,933 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 17:55:39,933 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:39,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:39,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:39,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:39,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,936 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:39,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 17:55:39,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,938 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,944 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 17:55:39,944 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:39,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:39,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:39,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:39,944 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 17:55:39,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 17:55:39,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:39,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:39,944 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:39,945 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:40,045 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:40,045 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,045 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:40,053 INFO [Listener at localhost/34581] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:40,054 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.StoreUtils(129): cannot split hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/47e2da4b3001473db24a569a64504200 because midkey is the same as first or last row 2023-02-09 17:55:40,054 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.StoreUtils(129): cannot split hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/9f5ddde55832491fad18894191f9361e because midkey is the same as first or last row 2023-02-09 17:55:40,054 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HRegion(2744): Flushing 2a3951a75479071f9743eaa175ca9ac6 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:55:40,065 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/eb52d7c62c1747f0b0e58d3f80127183 2023-02-09 17:55:40,076 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/8319d66f036342369a61dbf2eede58ba 2023-02-09 17:55:40,079 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf1/eb52d7c62c1747f0b0e58d3f80127183 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/eb52d7c62c1747f0b0e58d3f80127183 2023-02-09 17:55:40,082 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/eb52d7c62c1747f0b0e58d3f80127183, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:40,083 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/.tmp/cf2/8319d66f036342369a61dbf2eede58ba as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/8319d66f036342369a61dbf2eede58ba 2023-02-09 17:55:40,086 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/8319d66f036342369a61dbf2eede58ba, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:40,087 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2a3951a75479071f9743eaa175ca9ac6 in 33ms, sequenceid=82, compaction requested=true 2023-02-09 17:55:40,088 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.HRegion(2445): Flush status journal for 2a3951a75479071f9743eaa175ca9ac6: 2023-02-09 17:55:40,088 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.StoreUtils(129): cannot split hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/47e2da4b3001473db24a569a64504200 because midkey is the same as first or last row 2023-02-09 17:55:40,088 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35803] regionserver.StoreUtils(129): cannot split hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/9f5ddde55832491fad18894191f9361e because midkey is the same as first or last row 2023-02-09 17:55:40,089 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:40,089 DEBUG [Listener at localhost/34581] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-09 17:55:40,089 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:40,089 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,089 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,090 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,090 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,090 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-09 17:55:40,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-09 17:55:40,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,093 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,093 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 17:55:40,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965340094"}]},"ts":"1675965340094"} 2023-02-09 17:55:40,095 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-09 17:55:40,096 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-09 17:55:40,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, UNASSIGN}] 2023-02-09 17:55:40,097 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, UNASSIGN 2023-02-09 17:55:40,097 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=2a3951a75479071f9743eaa175ca9ac6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:40,098 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675965340097"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965340097"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965340097"}]},"ts":"1675965340097"} 2023-02-09 17:55:40,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 2a3951a75479071f9743eaa175ca9ac6, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:40,194 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,194 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 17:55:40,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:40,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2a3951a75479071f9743eaa175ca9ac6, disabling compactions & flushes 2023-02-09 17:55:40,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:40,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:40,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. after waiting 0 ms 2023-02-09 17:55:40,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:40,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:55:40,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:40,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6. 2023-02-09 17:55:40,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2a3951a75479071f9743eaa175ca9ac6: Waiting for close lock at 1675965340249Running coprocessor pre-close hooks at 1675965340249Disabling compacts and flushes for region at 1675965340249Disabling writes for close at 1675965340249Writing region close event to WAL at 1675965340250 (+1 ms)Running coprocessor post-close hooks at 1675965340254 (+4 ms)Closed at 1675965340254 2023-02-09 17:55:40,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:40,255 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=2a3951a75479071f9743eaa175ca9ac6, regionState=CLOSED 2023-02-09 17:55:40,256 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675965340255"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965340255"}]},"ts":"1675965340255"} 2023-02-09 17:55:40,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-09 17:55:40,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 2a3951a75479071f9743eaa175ca9ac6, server=jenkins-hbase4.apache.org,35803,1675965244552 in 158 msec 2023-02-09 17:55:40,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-09 17:55:40,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=2a3951a75479071f9743eaa175ca9ac6, UNASSIGN in 162 msec 2023-02-09 17:55:40,260 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965340260"}]},"ts":"1675965340260"} 2023-02-09 17:55:40,260 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-09 17:55:40,262 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-09 17:55:40,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 171 msec 2023-02-09 17:55:40,395 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,395 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 17:55:40,396 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-09 17:55:40,396 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,396 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,397 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-09 17:55:40,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,399 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,399 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,399 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,399 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 17:55:40,402 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:40,404 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/recovered.edits] 2023-02-09 17:55:40,407 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/47e2da4b3001473db24a569a64504200 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/47e2da4b3001473db24a569a64504200 2023-02-09 17:55:40,408 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6771b5fabba440348655229f86a94d7d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6771b5fabba440348655229f86a94d7d 2023-02-09 17:55:40,409 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6af20766c4984fe38dddd4e1a26aa8ba to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/6af20766c4984fe38dddd4e1a26aa8ba 2023-02-09 17:55:40,410 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/78df2e682a964413b6048e796f0b0b9e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/78df2e682a964413b6048e796f0b0b9e 2023-02-09 17:55:40,411 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/7c2f3037e677487f89a714158c17d1c8 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/7c2f3037e677487f89a714158c17d1c8 2023-02-09 17:55:40,412 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/afbaa9ba7849474a89c66d86c07d55c8 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/afbaa9ba7849474a89c66d86c07d55c8 2023-02-09 17:55:40,413 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/eb52d7c62c1747f0b0e58d3f80127183 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf1/eb52d7c62c1747f0b0e58d3f80127183 2023-02-09 17:55:40,415 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/32718f1ee8f4401bbba70ca0b0e60eb0 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/32718f1ee8f4401bbba70ca0b0e60eb0 2023-02-09 17:55:40,416 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/50412757b6de4b1ba520da9e8eefce6d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/50412757b6de4b1ba520da9e8eefce6d 2023-02-09 17:55:40,417 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/8319d66f036342369a61dbf2eede58ba to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/8319d66f036342369a61dbf2eede58ba 2023-02-09 17:55:40,417 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/96bf4b8a6c864bd4ba51caf4795ab8b7 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/96bf4b8a6c864bd4ba51caf4795ab8b7 2023-02-09 17:55:40,418 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/9f5ddde55832491fad18894191f9361e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/9f5ddde55832491fad18894191f9361e 2023-02-09 17:55:40,419 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/a557be1645224bc7b7f8f83103c82c1f to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/a557be1645224bc7b7f8f83103c82c1f 2023-02-09 17:55:40,420 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/d2c076031487403e849f844fd6d20b0a to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/cf2/d2c076031487403e849f844fd6d20b0a 2023-02-09 17:55:40,423 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/recovered.edits/85.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6/recovered.edits/85.seqid 2023-02-09 17:55:40,423 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testInvalidSplitKey/2a3951a75479071f9743eaa175ca9ac6 2023-02-09 17:55:40,423 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 17:55:40,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,426 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-09 17:55:40,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-09 17:55:40,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-09 17:55:40,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965340430"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:40,432 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 17:55:40,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2a3951a75479071f9743eaa175ca9ac6, NAME => 'testInvalidSplitKey,,1675965338435.2a3951a75479071f9743eaa175ca9ac6.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 17:55:40,432 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-09 17:55:40,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965340432"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:40,433 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-09 17:55:40,434 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:40,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 37 msec 2023-02-09 17:55:40,500 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,500 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 17:55:40,501 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-09 17:55:40,514 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 347) - Thread LEAK? -, OpenFileDescriptor=745 (was 653) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 183), ProcessCount=168 (was 168), AvailableMemoryMB=2987 (was 3084) 2023-02-09 17:55:40,527 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=177, ProcessCount=168, AvailableMemoryMB=2986 2023-02-09 17:55:40,527 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:40,527 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:40,527 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:40,529 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:55:40,530 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:55:40,531 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:40,532 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:55:40,532 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:55:40,534 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,534 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 empty. 2023-02-09 17:55:40,535 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,535 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 17:55:40,543 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 17:55:40,544 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 54ca31a1a82ef66e2bb6e7bc49e77ca2, NAME => 'testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:55:40,550 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:40,550 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 54ca31a1a82ef66e2bb6e7bc49e77ca2, disabling compactions & flushes 2023-02-09 17:55:40,550 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,550 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,550 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. after waiting 0 ms 2023-02-09 17:55:40,550 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,550 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,550 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: Waiting for close lock at 1675965340550Disabling compacts and flushes for region at 1675965340550Disabling writes for close at 1675965340550Writing region close event to WAL at 1675965340550Closed at 1675965340550 2023-02-09 17:55:40,552 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:55:40,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965340552"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965340552"}]},"ts":"1675965340552"} 2023-02-09 17:55:40,553 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:55:40,554 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:55:40,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965340554"}]},"ts":"1675965340554"} 2023-02-09 17:55:40,555 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 17:55:40,559 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:55:40,560 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:55:40,560 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:55:40,560 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:55:40,560 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:55:40,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, ASSIGN}] 2023-02-09 17:55:40,561 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, ASSIGN 2023-02-09 17:55:40,561 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:55:40,712 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:55:40,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:40,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965340713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965340713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965340713"}]},"ts":"1675965340713"} 2023-02-09 17:55:40,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:40,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 54ca31a1a82ef66e2bb6e7bc49e77ca2, NAME => 'testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:55:40,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,870 INFO [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,871 DEBUG [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1 2023-02-09 17:55:40,872 DEBUG [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1 2023-02-09 17:55:40,872 INFO [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 54ca31a1a82ef66e2bb6e7bc49e77ca2 columnFamilyName cf1 2023-02-09 17:55:40,872 INFO [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] regionserver.HStore(310): Store=54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:40,872 INFO [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,873 DEBUG [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2 2023-02-09 17:55:40,874 DEBUG [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2 2023-02-09 17:55:40,874 INFO [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 54ca31a1a82ef66e2bb6e7bc49e77ca2 columnFamilyName cf2 2023-02-09 17:55:40,874 INFO [StoreOpener-54ca31a1a82ef66e2bb6e7bc49e77ca2-1] regionserver.HStore(310): Store=54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:40,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:40,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:40,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:55:40,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 54ca31a1a82ef66e2bb6e7bc49e77ca2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64280514, jitterRate=-0.04214569926261902}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:40,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: Running coprocessor pre-open hook at 1675965340869Writing region info on filesystem at 1675965340869Initializing all the Stores at 1675965340870 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965340870Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965340870Cleaning up temporary data from old regions at 1675965340876 (+6 ms)Cleaning up detritus from prior splits at 1675965340876Running coprocessor post-open hooks at 1675965340879 (+3 ms)Region opened successfully at 1675965340879 2023-02-09 17:55:40,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2., pid=111, masterSystemTime=1675965340866 2023-02-09 17:55:40,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:40,881 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:40,881 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965340881"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965340881"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965340881"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965340881"}]},"ts":"1675965340881"} 2023-02-09 17:55:40,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-09 17:55:40,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 in 167 msec 2023-02-09 17:55:40,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-09 17:55:40,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, ASSIGN in 323 msec 2023-02-09 17:55:40,886 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:55:40,886 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965340886"}]},"ts":"1675965340886"} 2023-02-09 17:55:40,887 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 17:55:40,890 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:55:40,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 360 msec 2023-02-09 17:55:41,037 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,037 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,038 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:41,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 17:55:41,039 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,039 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,039 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 17:55:41,039 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:55:41,039 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,040 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,041 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,041 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,041 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,041 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,041 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,041 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,042 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,043 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,043 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,043 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,043 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,043 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:41,043 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,043 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:41,043 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. started... 2023-02-09 17:55:41,044 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:55:41,044 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,052 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/92126f40b85f44a093b89234a6b6d73b 2023-02-09 17:55:41,062 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/6d86612bdaa44394aa833cb8f63e0454 2023-02-09 17:55:41,066 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/92126f40b85f44a093b89234a6b6d73b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b 2023-02-09 17:55:41,069 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:41,070 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/6d86612bdaa44394aa833cb8f63e0454 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 2023-02-09 17:55:41,073 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:41,074 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 30ms, sequenceid=6, compaction requested=false 2023-02-09 17:55:41,074 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,075 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: 2023-02-09 17:55:41,075 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,075 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:41,075 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:41,075 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,075 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,075 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,076 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,076 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,077 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,077 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 17:55:41,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5719456e[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:41,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:55:41,079 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,079 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,079 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,079 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,080 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:41,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,080 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:41,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,083 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,083 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,083 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,084 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,083 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,084 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,084 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:41,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:55:41,087 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 17:55:41,087 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:41,087 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:41,087 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,087 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,089 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,089 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,090 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,091 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,091 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,092 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,094 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:41,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,095 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,095 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:41,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,094 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,095 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,195 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:41,195 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,195 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:41,205 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,205 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,206 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:41,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:41,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 17:55:41,207 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,207 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,207 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 17:55:41,207 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:55:41,208 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,208 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,209 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,209 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,209 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,211 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,211 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,211 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,211 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,211 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. started... 2023-02-09 17:55:41,211 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:41,213 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,214 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,214 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,219 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/295ae0a916e647448b9cae07c14de92c 2023-02-09 17:55:41,229 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/b85a46f5e6e64a1a808b3bfb207137da 2023-02-09 17:55:41,232 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/295ae0a916e647448b9cae07c14de92c as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c 2023-02-09 17:55:41,235 ERROR [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 38633d84-bd4c-44ba-82f9-206cacf2085d) service to localhost/127.0.0.1:38249 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:55:41,235 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:41,236 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/b85a46f5e6e64a1a808b3bfb207137da as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da 2023-02-09 17:55:41,239 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:41,240 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 29ms, sequenceid=19, compaction requested=false 2023-02-09 17:55:41,240 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: 2023-02-09 17:55:41,240 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,245 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,245 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 17:55:41,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bbe1206[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:41,247 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:55:41,247 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,249 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:41,250 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,250 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,250 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,250 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,252 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:41,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:55:41,254 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 17:55:41,254 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:41,254 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:41,254 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,254 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,257 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,258 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,258 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,260 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,260 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,260 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,262 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,263 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,262 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,262 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,263 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,363 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:41,363 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,363 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:41,374 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,374 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,374 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:41,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:41,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 17:55:41,376 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,376 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,376 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 17:55:41,376 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:55:41,376 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,376 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,377 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,377 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,377 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,377 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,378 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,378 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,379 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,380 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:41,380 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:41,380 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,380 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. started... 2023-02-09 17:55:41,380 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:41,381 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,382 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,382 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,382 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,383 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,383 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,388 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/c0727b31f14041b8a082b45e6a6698a1 2023-02-09 17:55:41,397 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/71ada4a5d3f24ee399db3e5068424f45 2023-02-09 17:55:41,400 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/c0727b31f14041b8a082b45e6a6698a1 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 2023-02-09 17:55:41,403 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:41,404 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/71ada4a5d3f24ee399db3e5068424f45 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 2023-02-09 17:55:41,407 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:41,407 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 27ms, sequenceid=32, compaction requested=true 2023-02-09 17:55:41,408 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: 2023-02-09 17:55:41,408 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,408 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:41,408 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:41,408 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,408 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,408 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,409 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,409 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,409 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,409 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 17:55:41,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b8bb1c1[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:41,411 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:55:41,411 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,412 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:41,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:41,412 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,413 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,413 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:41,414 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,414 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,414 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,414 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,415 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,415 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,416 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,416 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,416 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,416 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:41,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:55:41,419 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 17:55:41,419 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:41,419 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:41,419 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,419 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,421 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,421 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,422 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,423 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,423 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,424 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,430 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,430 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,430 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:41,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,430 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,431 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,431 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,431 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,431 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:41,431 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:41,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,531 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:41,531 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,531 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:41,542 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,542 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:41,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:41,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 17:55:41,544 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,544 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,544 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 17:55:41,544 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:55:41,544 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,544 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,545 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,545 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,545 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,545 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,546 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,546 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,548 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:41,548 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:41,548 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,547 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,548 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. started... 2023-02-09 17:55:41,548 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,548 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,548 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,548 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:41,549 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,550 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,550 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,550 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,550 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,550 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,556 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/cedebaaeeb2046e0a7e11b71ee8d899e 2023-02-09 17:55:41,566 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/1cda9ca05a4548c5afb3f9155bad0c94 2023-02-09 17:55:41,569 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/cedebaaeeb2046e0a7e11b71ee8d899e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e 2023-02-09 17:55:41,572 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:41,572 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/1cda9ca05a4548c5afb3f9155bad0c94 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 2023-02-09 17:55:41,576 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:41,576 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 28ms, sequenceid=45, compaction requested=true 2023-02-09 17:55:41,576 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: 2023-02-09 17:55:41,577 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,577 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:41,577 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:41,577 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,577 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,577 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,579 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,579 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,580 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,580 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 17:55:41,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cb87cd3[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:41,581 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:55:41,581 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,582 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,583 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,583 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:41,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:41,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:41,584 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,584 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,584 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,584 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,585 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,585 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,585 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,585 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,585 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,585 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:41,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:55:41,588 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 17:55:41,588 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:41,588 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:41,588 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,588 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:41,590 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,590 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,590 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,591 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,592 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,593 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,593 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,594 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,596 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,596 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,597 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,697 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:41,697 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,697 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:41,708 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,708 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,709 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:41,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:41,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 17:55:41,710 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,710 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,710 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 17:55:41,710 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:55:41,711 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,711 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,712 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,712 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,712 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,712 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,714 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,714 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. started... 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:41,714 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,715 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,717 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,717 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,717 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,717 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,717 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,722 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/03fb677fe0614caebbb48fd1c6655b0d 2023-02-09 17:55:41,732 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/0e4206eb1d1f456cb3240b6c0549ed0e 2023-02-09 17:55:41,735 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/03fb677fe0614caebbb48fd1c6655b0d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d 2023-02-09 17:55:41,738 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:41,738 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/0e4206eb1d1f456cb3240b6c0549ed0e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e 2023-02-09 17:55:41,741 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:41,742 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 28ms, sequenceid=58, compaction requested=true 2023-02-09 17:55:41,742 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: 2023-02-09 17:55:41,742 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,750 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,750 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,750 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,750 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 17:55:41,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4117601f[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:41,752 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:55:41,752 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:41,753 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,754 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,754 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:41,754 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,754 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:41,755 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:41,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:55:41,759 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 17:55:41,759 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:41,759 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:41,759 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,759 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,761 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,761 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,762 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,763 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,764 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,765 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,765 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,768 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,768 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,769 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,869 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:41,869 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,869 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:41,880 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:41,880 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:41,880 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:41,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:41,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 17:55:41,884 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,884 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,884 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 17:55:41,884 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 17:55:41,884 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,884 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,886 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,886 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,886 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,886 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,887 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,887 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,888 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 17:55:41,888 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,888 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,888 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,888 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,888 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:41,889 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,889 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. started... 2023-02-09 17:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,889 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,897 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/eaa421280be34203bd3d7b3b37bc2824 2023-02-09 17:55:41,910 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/8d4fa58b091b42ddb6dd28e21c2d640e 2023-02-09 17:55:41,914 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/eaa421280be34203bd3d7b3b37bc2824 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 2023-02-09 17:55:41,917 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:41,918 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/8d4fa58b091b42ddb6dd28e21c2d640e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e 2023-02-09 17:55:41,921 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:41,922 INFO [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 33ms, sequenceid=71, compaction requested=true 2023-02-09 17:55:41,922 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: 2023-02-09 17:55:41,922 DEBUG [rs(jenkins-hbase4.apache.org,35803,1675965244552)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:41,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:41,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:41,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 17:55:41,922 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 17:55:41,923 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,923 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,924 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,924 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 17:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a226419[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:41,925 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 17:55:41,926 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,927 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,928 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,928 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:41,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 17:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:41,929 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,929 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,929 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,930 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 17:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 17:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,35803,1675965244552': 2023-02-09 17:55:41,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 17:55:41,933 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 17:55:41,933 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:41,933 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:41,933 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 17:55:41,933 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,934 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,935 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:41,937 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 17:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,943 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 17:55:41,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:41,943 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:41,943 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,943 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:41,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:41,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 17:55:41,944 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 17:55:42,045 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:42,045 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:42,045 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:42,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:42,303 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 17:55:42,304 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:42,305 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,307 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:42,307 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:42,307 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:42,555 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:42,556 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:42,556 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:42,557 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:42,558 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 17:55:42,558 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:42,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:42,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:42,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:42,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:42,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:42,559 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:42,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:42,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:42,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:42,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:42,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:42,560 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:42,561 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:42,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:42,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:42,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:42,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:42,562 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:42,563 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:42,564 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:42,564 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,564 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:42,564 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,564 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,564 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,564 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 17:55:42,565 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:42,565 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:42,566 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965372566 2023-02-09 17:55:42,566 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:42,567 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,569 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:42,570 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:42,570 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:42,570 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965402570 2023-02-09 17:55:42,570 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965462570 2023-02-09 17:55:42,570 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:55:42,570 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:42,572 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:42,572 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:42,572 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:42,820 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,820 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:42,820 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:42,820 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:42,820 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:42,821 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:42,821 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:42,821 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:42,821 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:42,821 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:42,823 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 17:55:42,823 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:42,823 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:42,824 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:42,824 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:42,824 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:42,824 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:42,825 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:42,826 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,827 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:42,828 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:42,829 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,829 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:42,829 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,829 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,829 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,829 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 17:55:42,829 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:42,830 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:42,830 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965372830 2023-02-09 17:55:42,831 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:42,832 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,834 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:42,834 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:42,834 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:42,834 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965402834 2023-02-09 17:55:42,834 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965462834 2023-02-09 17:55:42,835 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:55:42,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:42,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN}] 2023-02-09 17:55:42,837 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:42,837 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:42,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:42,837 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:43,085 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,085 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:43,085 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:43,085 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:43,085 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:43,086 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:43,086 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:43,086 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:43,086 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:43,086 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:43,088 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 17:55:43,088 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:43,088 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:43,088 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:43,089 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:43,089 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,089 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:43,089 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,089 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:43,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,090 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:43,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:43,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,091 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,092 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:43,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,093 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:43,094 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,094 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:43,094 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,094 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,095 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,095 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 17:55:43,095 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:43,096 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:43,096 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965373096 2023-02-09 17:55:43,096 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:43,097 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,100 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:43,100 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:43,100 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:43,100 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965403100 2023-02-09 17:55:43,100 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965463100 2023-02-09 17:55:43,100 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:55:43,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN}] 2023-02-09 17:55:43,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:43,102 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:43,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:43,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965343103"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965343103"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965343103"}]},"ts":"1675965343103"} 2023-02-09 17:55:43,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:43,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 54ca31a1a82ef66e2bb6e7bc49e77ca2, disabling compactions & flushes 2023-02-09 17:55:43,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:43,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:43,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. after waiting 0 ms 2023-02-09 17:55:43,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:43,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:55:43,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/6699009e480948c9847484046a2979d5 2023-02-09 17:55:43,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/19bcc43bbf5d4418a36f6512f80b74d3 2023-02-09 17:55:43,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf1/6699009e480948c9847484046a2979d5 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 2023-02-09 17:55:43,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:43,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/.tmp/cf2/19bcc43bbf5d4418a36f6512f80b74d3 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 2023-02-09 17:55:43,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:43,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 33ms, sequenceid=82, compaction requested=true 2023-02-09 17:55:43,293 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:55:43,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:43,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. 2023-02-09 17:55:43,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 54ca31a1a82ef66e2bb6e7bc49e77ca2: Waiting for close lock at 1675965343256Running coprocessor pre-close hooks at 1675965343256Disabling compacts and flushes for region at 1675965343256Disabling writes for close at 1675965343256Obtaining lock to block concurrent updates at 1675965343256Preparing flush snapshotting stores in 54ca31a1a82ef66e2bb6e7bc49e77ca2 at 1675965343256Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675965343256Flushing stores of testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. at 1675965343257 (+1 ms)Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1: creating writer at 1675965343258 (+1 ms)Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1: appending metadata at 1675965343259 (+1 ms)Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1: closing flushed file at 1675965343259Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2: creating writer at 1675965343269 (+10 ms)Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2: appending metadata at 1675965343270 (+1 ms)Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2: closing flushed file at 1675965343270Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1: reopening flushed file at 1675965343282 (+12 ms)Flushing 54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2: reopening flushed file at 1675965343286 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 54ca31a1a82ef66e2bb6e7bc49e77ca2 in 33ms, sequenceid=82, compaction requested=true at 1675965343290 (+4 ms)Writing region close event to WAL at 1675965343291 (+1 ms)Running coprocessor post-close hooks at 1675965343294 (+3 ms)Closed at 1675965343295 (+1 ms) 2023-02-09 17:55:43,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED 2023-02-09 17:55:43,297 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965343297"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965343297"}]},"ts":"1675965343297"} 2023-02-09 17:55:43,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 17:55:43,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 in 194 msec 2023-02-09 17:55:43,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 17:55:43,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN in 199 msec 2023-02-09 17:55:43,302 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:43,302 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:43,303 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:43,350 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,350 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:43,350 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:43,350 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:43,351 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:43,351 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:43,351 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:43,351 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:43,351 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:43,351 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:43,352 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 17:55:43,353 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:43,353 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:43,353 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:43,353 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:43,353 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:43,353 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:43,354 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,355 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,355 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,355 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:43,355 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:43,356 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,356 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,356 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,356 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:43,357 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,357 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,357 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,357 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:43,357 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,358 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,358 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,358 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,358 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:43,359 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,359 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,359 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,359 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,359 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,360 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:43,360 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,360 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:43,360 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:43,360 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:43,360 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,360 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,360 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,360 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:43,361 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:43,361 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-09 17:55:43,361 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:43,362 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:43,362 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965373362 2023-02-09 17:55:43,362 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:43,363 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,365 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:43,366 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:43,366 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:43,366 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965403366 2023-02-09 17:55:43,366 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965463366 2023-02-09 17:55:43,366 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:55:43,368 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:43,376 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, threads=14 2023-02-09 17:55:43,376 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,376 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,376 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,376 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,377 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,378 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,378 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,475 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,485 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,511 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,524 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,672 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,672 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,676 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,684 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,686 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,686 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,688 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,688 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,688 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,689 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,689 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 54ca31a1a82ef66e2bb6e7bc49e77ca2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:55:43,721 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:55:43,728 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:43,728 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:43,728 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:43,866 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,866 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:43,866 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:43,866 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:43,867 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:43,867 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:43,867 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:43,867 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:43,867 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:43,868 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:43,869 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 17:55:43,869 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:43,869 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:43,869 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:43,870 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:43,870 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:43,870 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:43,870 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,871 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:43,871 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,871 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:43,871 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,871 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:43,872 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,872 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:43,872 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,872 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:43,873 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:43,873 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,873 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:43,873 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,874 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:43,874 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,874 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:43,874 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,874 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:43,875 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,875 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:43,875 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,875 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,876 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:43,876 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,876 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:43,876 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,876 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,877 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:43,877 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:43,877 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,877 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:43,877 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:43,878 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:43,878 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,878 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,878 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,878 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:43,878 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:43,878 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 17:55:43,879 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:43,879 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:43,880 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965373880 2023-02-09 17:55:43,881 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:43,881 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:43,883 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:43,883 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:43,884 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:43,884 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965403884 2023-02-09 17:55:43,884 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965463884 2023-02-09 17:55:43,884 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 17:55:43,893 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, threads=14 2023-02-09 17:55:43,894 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,900 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,900 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,900 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,901 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,902 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,903 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,903 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,969 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,970 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,972 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,972 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,981 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,982 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,983 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,986 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,987 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,989 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,989 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,990 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,990 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,991 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 for region: 54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:43,991 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 54ca31a1a82ef66e2bb6e7bc49e77ca2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:55:44,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:44,024 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:55:44,025 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:55:44,025 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:44,025 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:44,025 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:44,134 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,134 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:44,134 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:44,134 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:44,134 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:44,135 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:44,135 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:44,135 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:44,135 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:44,135 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:44,136 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 17:55:44,136 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:44,136 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:44,137 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:44,137 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:44,137 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:44,137 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:44,137 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,138 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,139 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,140 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:44,140 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,140 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,140 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,140 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,140 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:44,141 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:44,142 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:44,142 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,142 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,142 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,142 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,142 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,142 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:55:44,142 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:44,143 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:44,143 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965374143 2023-02-09 17:55:44,145 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:44,145 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,148 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,148 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,148 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:44,148 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965404148 2023-02-09 17:55:44,148 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965464148 2023-02-09 17:55:44,148 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 17:55:44,150 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:44,152 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:44,152 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:44,152 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:44,235 ERROR [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 634cd12e-a559-4a0e-a30b-33f9304f450f) service to localhost/127.0.0.1:38249 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:55:44,398 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,398 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:44,398 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:44,399 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:44,399 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:44,399 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:44,399 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:44,399 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:44,399 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:44,400 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:44,401 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,402 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,403 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,404 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,405 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:44,406 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,406 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:44,406 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:44,406 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:44,406 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,406 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,406 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,406 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,406 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,406 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:55:44,407 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:44,407 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:44,408 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965374408 2023-02-09 17:55:44,409 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:44,409 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,412 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,412 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,412 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:44,412 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965404412 2023-02-09 17:55:44,412 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965464412 2023-02-09 17:55:44,412 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:55:44,412 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:44,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965344414"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675965344414"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675965344414"}]},"ts":"1675965344414"} 2023-02-09 17:55:44,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965344414"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965344414"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965344414"}]},"ts":"1675965344414"} 2023-02-09 17:55:44,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965344414"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965344414"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965344414"}]},"ts":"1675965344414"} 2023-02-09 17:55:44,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:44,416 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:44,416 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:44,662 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,662 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:44,662 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:44,663 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:44,663 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:44,663 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:44,663 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:44,663 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:44,663 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:44,664 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:44,665 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 17:55:44,665 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:44,665 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:44,665 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,666 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,667 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,667 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:44,667 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,667 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,667 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,668 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:44,668 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:44,668 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,668 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,668 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,668 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,669 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,670 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:44,671 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,671 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,671 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,671 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,671 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 17:55:44,672 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:44,672 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:44,673 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965374673 2023-02-09 17:55:44,674 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:44,674 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,676 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,676 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,677 INFO [Listener at localhost/34581] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552, table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:44,677 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 17:55:44,677 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 17:55:44,677 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:44,677 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965404677 2023-02-09 17:55:44,677 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965464677 2023-02-09 17:55:44,677 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:55:44,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675965344677"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675965344677"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675965344677"}]},"ts":"1675965344677"} 2023-02-09 17:55:44,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965344677"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965344677"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965344677"}]},"ts":"1675965344677"} 2023-02-09 17:55:44,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965344677"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965344677"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965344677"}]},"ts":"1675965344677"} 2023-02-09 17:55:44,679 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:44,681 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:44,681 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:44,681 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:44,927 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,927 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:44,928 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:44,928 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:44,928 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:44,928 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:44,928 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:44,928 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:44,928 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:44,929 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:44,930 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-09 17:55:44,930 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:44,930 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:44,930 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:44,930 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:44,930 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:44,931 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:44,931 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,931 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:44,931 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,931 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:44,932 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,932 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:44,932 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,932 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:44,932 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,933 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:44,933 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:44,933 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,933 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:44,933 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,934 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:44,935 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,935 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,935 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:44,935 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,935 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:44,935 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:44,936 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:44,936 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,937 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,937 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,937 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,937 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:44,937 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 17:55:44,937 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:44,938 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:44,938 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965374938 2023-02-09 17:55:44,938 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:44,940 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,942 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,942 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:44,942 INFO [Listener at localhost/34581] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552, table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:44,942 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 17:55:44,942 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 17:55:44,942 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:44,942 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965404942 2023-02-09 17:55:44,942 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965464942 2023-02-09 17:55:44,942 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:55:44,943 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:44,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN}] 2023-02-09 17:55:44,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:44,944 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:44,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:44,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:45,193 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:45,193 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:45,194 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:45,195 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-09 17:55:45,195 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:45,195 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:45,195 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:45,196 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:45,197 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:45,198 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:45,199 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:45,200 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,200 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:45,200 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,200 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:45,200 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,200 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,200 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,200 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,200 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,200 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 17:55:45,201 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:45,201 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:45,202 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965375202 2023-02-09 17:55:45,202 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:45,203 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,205 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:45,205 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:45,206 INFO [Listener at localhost/34581] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552, table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:45,206 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 17:55:45,206 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 17:55:45,206 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:45,206 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965405206 2023-02-09 17:55:45,206 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965465206 2023-02-09 17:55:45,206 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 17:55:45,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN}] 2023-02-09 17:55:45,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:45,209 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN 2023-02-09 17:55:45,209 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:55:45,210 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN 2023-02-09 17:55:45,210 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552; forceNewPlan=false, retain=false 2023-02-09 17:55:45,359 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:55:45,361 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,361 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965345361"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965345361"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965345361"}]},"ts":"1675965345361"} 2023-02-09 17:55:45,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 6a8a6d1495885b8e0f6a82d61bb7547c, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:45,364 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,364 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965345364"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965345364"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965345364"}]},"ts":"1675965345364"} 2023-02-09 17:55:45,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 4ac1d544fc33f59c1bf380e57c9b7c84, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:45,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a8a6d1495885b8e0f6a82d61bb7547c, NAME => 'testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 17:55:45,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:45,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:45,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,518 INFO [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,519 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1 2023-02-09 17:55:45,519 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1 2023-02-09 17:55:45,519 INFO [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a8a6d1495885b8e0f6a82d61bb7547c columnFamilyName cf1 2023-02-09 17:55:45,529 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d-top 2023-02-09 17:55:45,532 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c-top 2023-02-09 17:55:45,535 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5-top 2023-02-09 17:55:45,538 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b-top 2023-02-09 17:55:45,541 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1-top 2023-02-09 17:55:45,544 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e-top 2023-02-09 17:55:45,547 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824-top 2023-02-09 17:55:45,547 INFO [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(310): Store=6a8a6d1495885b8e0f6a82d61bb7547c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:45,547 INFO [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,548 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2 2023-02-09 17:55:45,548 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2 2023-02-09 17:55:45,548 INFO [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a8a6d1495885b8e0f6a82d61bb7547c columnFamilyName cf2 2023-02-09 17:55:45,559 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e-top 2023-02-09 17:55:45,561 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3-top 2023-02-09 17:55:45,566 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94-top 2023-02-09 17:55:45,569 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454-top 2023-02-09 17:55:45,573 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45-top 2023-02-09 17:55:45,575 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e-top 2023-02-09 17:55:45,578 DEBUG [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da-top 2023-02-09 17:55:45,578 INFO [StoreOpener-6a8a6d1495885b8e0f6a82d61bb7547c-1] regionserver.HStore(310): Store=6a8a6d1495885b8e0f6a82d61bb7547c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:45,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:45,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6a8a6d1495885b8e0f6a82d61bb7547c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64237586, jitterRate=-0.04278537631034851}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:45,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6a8a6d1495885b8e0f6a82d61bb7547c: Running coprocessor pre-open hook at 1675965345517Writing region info on filesystem at 1675965345517Initializing all the Stores at 1675965345517Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965345518 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965345518Cleaning up temporary data from old regions at 1675965345580 (+62 ms)Cleaning up detritus from prior splits at 1675965345581 (+1 ms)Running coprocessor post-open hooks at 1675965345583 (+2 ms)Region opened successfully at 1675965345583 2023-02-09 17:55:45,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c., pid=117, masterSystemTime=1675965345514 2023-02-09 17:55:45,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:45,586 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:45,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:45,586 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:45,586 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,587 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,587 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1912): 6a8a6d1495885b8e0f6a82d61bb7547c/cf1 is initiating minor compaction (all files) 2023-02-09 17:55:45,587 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1912): 6a8a6d1495885b8e0f6a82d61bb7547c/cf2 is initiating minor compaction (all files) 2023-02-09 17:55:45,587 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a8a6d1495885b8e0f6a82d61bb7547c/cf1 in testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,587 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a8a6d1495885b8e0f6a82d61bb7547c/cf2 in testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,587 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5-top] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/.tmp, totalSize=34.9 K 2023-02-09 17:55:45,588 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3-top] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/.tmp, totalSize=34.9 K 2023-02-09 17:55:45,588 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965341033 2023-02-09 17:55:45,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,588 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965341033 2023-02-09 17:55:45,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:45,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4ac1d544fc33f59c1bf380e57c9b7c84, NAME => 'testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 17:55:45,588 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965341196 2023-02-09 17:55:45,589 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965341196 2023-02-09 17:55:45,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:45,589 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:45,589 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965345589"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965345589"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965345589"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965345589"}]},"ts":"1675965345589"} 2023-02-09 17:55:45,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,589 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965341365 2023-02-09 17:55:45,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,589 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965341365 2023-02-09 17:55:45,590 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965341533 2023-02-09 17:55:45,590 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965341533 2023-02-09 17:55:45,590 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675965341699 2023-02-09 17:55:45,590 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675965341699 2023-02-09 17:55:45,591 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675965341871 2023-02-09 17:55:45,591 INFO [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,591 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675965341871 2023-02-09 17:55:45,591 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965342047 2023-02-09 17:55:45,592 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965342047 2023-02-09 17:55:45,593 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1 2023-02-09 17:55:45,593 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1 2023-02-09 17:55:45,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-09 17:55:45,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 6a8a6d1495885b8e0f6a82d61bb7547c, server=jenkins-hbase4.apache.org,35803,1675965244552 in 228 msec 2023-02-09 17:55:45,593 INFO [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ac1d544fc33f59c1bf380e57c9b7c84 columnFamilyName cf1 2023-02-09 17:55:45,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN in 386 msec 2023-02-09 17:55:45,604 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] throttle.PressureAwareThroughputController(145): 6a8a6d1495885b8e0f6a82d61bb7547c#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:45,605 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6a8a6d1495885b8e0f6a82d61bb7547c#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:45,609 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d-bottom 2023-02-09 17:55:45,611 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c-bottom 2023-02-09 17:55:45,621 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5-bottom 2023-02-09 17:55:45,626 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b-bottom 2023-02-09 17:55:45,629 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/.tmp/cf2/84ac42ac9aaf404aa1800080c301dc35 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/84ac42ac9aaf404aa1800080c301dc35 2023-02-09 17:55:45,630 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1-bottom 2023-02-09 17:55:45,632 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/.tmp/cf1/2cc0a90452ea47d6bd346ff73b27632d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/2cc0a90452ea47d6bd346ff73b27632d 2023-02-09 17:55:45,634 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e-bottom 2023-02-09 17:55:45,635 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6a8a6d1495885b8e0f6a82d61bb7547c/cf2 of 6a8a6d1495885b8e0f6a82d61bb7547c into 84ac42ac9aaf404aa1800080c301dc35(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:45,635 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a8a6d1495885b8e0f6a82d61bb7547c: 2023-02-09 17:55:45,635 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c., storeName=6a8a6d1495885b8e0f6a82d61bb7547c/cf2, priority=9, startTime=1675965345586; duration=0sec 2023-02-09 17:55:45,635 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:45,637 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6a8a6d1495885b8e0f6a82d61bb7547c/cf1 of 6a8a6d1495885b8e0f6a82d61bb7547c into 2cc0a90452ea47d6bd346ff73b27632d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:45,637 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a8a6d1495885b8e0f6a82d61bb7547c: 2023-02-09 17:55:45,637 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c., storeName=6a8a6d1495885b8e0f6a82d61bb7547c/cf1, priority=9, startTime=1675965345583; duration=0sec 2023-02-09 17:55:45,637 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:45,638 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824-bottom 2023-02-09 17:55:45,638 INFO [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(310): Store=4ac1d544fc33f59c1bf380e57c9b7c84/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:45,638 INFO [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,639 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2 2023-02-09 17:55:45,639 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2 2023-02-09 17:55:45,639 INFO [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ac1d544fc33f59c1bf380e57c9b7c84 columnFamilyName cf2 2023-02-09 17:55:45,649 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e-bottom 2023-02-09 17:55:45,652 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3-bottom 2023-02-09 17:55:45,654 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94-bottom 2023-02-09 17:55:45,657 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454-bottom 2023-02-09 17:55:45,660 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45-bottom 2023-02-09 17:55:45,662 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e-bottom 2023-02-09 17:55:45,665 DEBUG [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da-bottom 2023-02-09 17:55:45,665 INFO [StoreOpener-4ac1d544fc33f59c1bf380e57c9b7c84-1] regionserver.HStore(310): Store=4ac1d544fc33f59c1bf380e57c9b7c84/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:45,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:45,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:45,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4ac1d544fc33f59c1bf380e57c9b7c84; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67996473, jitterRate=0.013226404786109924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:45,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4ac1d544fc33f59c1bf380e57c9b7c84: Running coprocessor pre-open hook at 1675965345589Writing region info on filesystem at 1675965345589Initializing all the Stores at 1675965345590 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965345591 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965345591Cleaning up temporary data from old regions at 1675965345667 (+76 ms)Cleaning up detritus from prior splits at 1675965345667Running coprocessor post-open hooks at 1675965345669 (+2 ms)Region opened successfully at 1675965345669 2023-02-09 17:55:45,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84., pid=118, masterSystemTime=1675965345514 2023-02-09 17:55:45,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:45,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:45,671 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:45,671 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:45,671 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,671 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1912): 4ac1d544fc33f59c1bf380e57c9b7c84/cf1 is initiating minor compaction (all files) 2023-02-09 17:55:45,671 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,671 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1912): 4ac1d544fc33f59c1bf380e57c9b7c84/cf2 is initiating minor compaction (all files) 2023-02-09 17:55:45,671 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4ac1d544fc33f59c1bf380e57c9b7c84/cf1 in testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,671 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4ac1d544fc33f59c1bf380e57c9b7c84/cf2 in testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,672 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/.tmp, totalSize=34.9 K 2023-02-09 17:55:45,672 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/.tmp, totalSize=34.9 K 2023-02-09 17:55:45,672 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965341033 2023-02-09 17:55:45,672 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965341033 2023-02-09 17:55:45,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,673 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965341196 2023-02-09 17:55:45,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:45,673 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965341196 2023-02-09 17:55:45,673 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965341365 2023-02-09 17:55:45,673 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,673 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965341365 2023-02-09 17:55:45,673 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965345673"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965345673"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965345673"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965345673"}]},"ts":"1675965345673"} 2023-02-09 17:55:45,673 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965341533 2023-02-09 17:55:45,674 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965341533 2023-02-09 17:55:45,674 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965341699 2023-02-09 17:55:45,674 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965341699 2023-02-09 17:55:45,674 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965341871 2023-02-09 17:55:45,674 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965341871 2023-02-09 17:55:45,674 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] compactions.Compactor(207): Compacting 6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965342047 2023-02-09 17:55:45,675 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] compactions.Compactor(207): Compacting 19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965342047 2023-02-09 17:55:45,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-09 17:55:45,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 4ac1d544fc33f59c1bf380e57c9b7c84, server=jenkins-hbase4.apache.org,35803,1675965244552 in 309 msec 2023-02-09 17:55:45,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-09 17:55:45,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN in 470 msec 2023-02-09 17:55:45,678 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 17:55:45,678 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:45,678 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 17:55:45,684 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4ac1d544fc33f59c1bf380e57c9b7c84#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:45,685 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] throttle.PressureAwareThroughputController(145): 4ac1d544fc33f59c1bf380e57c9b7c84#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:45,702 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/.tmp/cf1/70c0f7e18e544adaaece127d38621f81 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/70c0f7e18e544adaaece127d38621f81 2023-02-09 17:55:45,704 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/.tmp/cf2/989f7426f14e45fa89d0f78cb8dc9747 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/989f7426f14e45fa89d0f78cb8dc9747 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:45,707 ERROR [Listener at localhost/34581] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 17:55:45,707 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4ac1d544fc33f59c1bf380e57c9b7c84/cf1 of 4ac1d544fc33f59c1bf380e57c9b7c84 into 70c0f7e18e544adaaece127d38621f81(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:45,707 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 17:55:45,708 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4ac1d544fc33f59c1bf380e57c9b7c84: 2023-02-09 17:55:45,708 INFO [Listener at localhost/34581] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 17:55:45,708 INFO [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84., storeName=4ac1d544fc33f59c1bf380e57c9b7c84/cf1, priority=9, startTime=1675965345670; duration=0sec 2023-02-09 17:55:45,708 DEBUG [RS:1;jenkins-hbase4:35803-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:45,708 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-09 17:55:45,710 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 17:55:45,711 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 17:55:45,711 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 17:55:45,711 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 17:55:45,712 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:45,712 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d08cc38916d551c13ac0de49fa30546d, daughterA=1d292a7367afe6611400f38936e4376b, daughterB=4b2902b446be90fa1b555fe9a2d595e6 2023-02-09 17:55:45,712 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:45,712 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 17:55:45,712 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:45,713 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=18021c0707f7eea71bc7fa8ec852264c, daughterA=a39cddc4336f63004ea998d92908d13f, daughterB=95d83923843793eee3680b2585b92089 2023-02-09 17:55:45,713 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:45,713 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 17:55:45,713 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:45,714 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3371e42e10cd6e1053fe33400fe25afb, daughterA=4aabcf7d5345e7c98b9b70b3983de444, daughterB=c7f34f4d98b171ae6edb78220c6f35c5 2023-02-09 17:55:45,714 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3371e42e10cd6e1053fe33400fe25afb, ASSIGN 2023-02-09 17:55:45,714 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:45,714 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 17:55:45,714 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:45,715 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2a988393b3c8e714357445e5243b5680, daughterA=dd3bd80bdb6a1eeb6189e07c0b4193a8, daughterB=130722f0711887fd4e2040a6ee000337 2023-02-09 17:55:45,715 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:45,715 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 17:55:45,715 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:45,715 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=4ef9f57187c5579008833c2bab1cfd4d, daughterA=1ee04d07a0c83d2cc0c0a9b53cab9a6d, daughterB=58cef6fb1a2ed5ac5e50a58a3e6d9dea 2023-02-09 17:55:45,715 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:45,716 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 17:55:45,716 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,716 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4ac1d544fc33f59c1bf380e57c9b7c84/cf2 of 4ac1d544fc33f59c1bf380e57c9b7c84 into 989f7426f14e45fa89d0f78cb8dc9747(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:45,716 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4ac1d544fc33f59c1bf380e57c9b7c84: 2023-02-09 17:55:45,716 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,716 INFO [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84., storeName=4ac1d544fc33f59c1bf380e57c9b7c84/cf2, priority=9, startTime=1675965345670; duration=0sec 2023-02-09 17:55:45,716 DEBUG [RS:1;jenkins-hbase4:35803-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:45,716 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43923,1675965244585, table=testRollbackForSplitTableRegionWithReplica, region=0389f4cb546df1d55fd1fdc67418f228 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=520e862302f7f9f2f3a223d4e5ede955, daughterA=cdd7151864707b29ef46651369ad0545, daughterB=57953e6dab1ee07750b75d82f2f51e99 2023-02-09 17:55:45,717 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,717 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 17:55:45,717 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:45,717 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:45,717 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 17:55:45,718 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:45,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN 2023-02-09 17:55:45,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN 2023-02-09 17:55:45,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN 2023-02-09 17:55:45,719 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 6a8a6d1495885b8e0f6a82d61bb7547c, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,720 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 4ac1d544fc33f59c1bf380e57c9b7c84, server=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:45,720 DEBUG [Listener at localhost/34581] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c held the lock before restarting, call acquireLock to restore it. 2023-02-09 17:55:45,720 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,720 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,720 INFO [Listener at localhost/34581] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,720 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,720 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 54ca31a1a82ef66e2bb6e7bc49e77ca2, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,720 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,721 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,721 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 6a8a6d1495885b8e0f6a82d61bb7547c, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,721 DEBUG [Listener at localhost/34581] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 4ac1d544fc33f59c1bf380e57c9b7c84, server=jenkins-hbase4.apache.org,35803,1675965244552 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 17:55:45,721 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-09 17:55:45,721 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 17:55:45,722 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,35803,1675965244552, table=hbase:meta, region=1588230740 2023-02-09 17:55:45,722 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675965375722 2023-02-09 17:55:45,725 DEBUG [Listener at localhost/34581] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 17:55:45,725 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:45,728 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=b77a8e95cd64fdcc1c76452d991420d2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:45,728 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=54ca31a1a82ef66e2bb6e7bc49e77ca2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=2 2023-02-09 17:55:45,728 INFO [Listener at localhost/34581] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,35803,1675965244552, table=testRecoveryAndDoubleExecution, region=54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:45,728 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=86 2023-02-09 17:55:45,729 DEBUG [Listener at localhost/34581] assignment.RegionStateStore(147): Load hbase:meta entry region=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35803,1675965244552, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552, openSeqNum=86 2023-02-09 17:55:45,729 INFO [Listener at localhost/34581] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 17:55:45,729 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675965405729 2023-02-09 17:55:45,729 INFO [Listener at localhost/34581] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675965465729 2023-02-09 17:55:45,729 INFO [Listener at localhost/34581] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-09 17:55:45,729 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 17:55:45,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=54ca31a1a82ef66e2bb6e7bc49e77ca2, daughterA=4ac1d544fc33f59c1bf380e57c9b7c84, daughterB=6a8a6d1495885b8e0f6a82d61bb7547c in 3.4240 sec 2023-02-09 17:55:45,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35803] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:57330 deadline: 1675965405980, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2. is not online on jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:46,091 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:55:46,097 INFO [Listener at localhost/34581] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 17:55:46,099 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:55:46,105 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:46,106 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,106 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,107 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,107 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,107 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-09 17:55:46,107 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-09 17:55:46,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,109 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,110 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,110 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965346110"}]},"ts":"1675965346110"} 2023-02-09 17:55:46,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 17:55:46,111 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 17:55:46,112 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-09 17:55:46,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, UNASSIGN}] 2023-02-09 17:55:46,114 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, UNASSIGN 2023-02-09 17:55:46,114 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:46,114 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965346114"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965346114"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965346114"}]},"ts":"1675965346114"} 2023-02-09 17:55:46,115 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 6a8a6d1495885b8e0f6a82d61bb7547c, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:46,117 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, UNASSIGN 2023-02-09 17:55:46,117 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:46,117 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965346117"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965346117"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965346117"}]},"ts":"1675965346117"} 2023-02-09 17:55:46,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 4ac1d544fc33f59c1bf380e57c9b7c84, server=jenkins-hbase4.apache.org,35803,1675965244552}] 2023-02-09 17:55:46,211 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,211 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 17:55:46,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:46,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6a8a6d1495885b8e0f6a82d61bb7547c, disabling compactions & flushes 2023-02-09 17:55:46,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:46,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:46,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. after waiting 0 ms 2023-02-09 17:55:46,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:46,267 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5-top] to archive 2023-02-09 17:55:46,268 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:46,270 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,271 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,272 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,273 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,274 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,275 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,276 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,277 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3-top] to archive 2023-02-09 17:55:46,278 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:46,279 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,280 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,281 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,282 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,282 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,283 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,284 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:55:46,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:46,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c. 2023-02-09 17:55:46,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6a8a6d1495885b8e0f6a82d61bb7547c: Waiting for close lock at 1675965346267Running coprocessor pre-close hooks at 1675965346267Disabling compacts and flushes for region at 1675965346267Disabling writes for close at 1675965346267Writing region close event to WAL at 1675965346285 (+18 ms)Running coprocessor post-close hooks at 1675965346288 (+3 ms)Closed at 1675965346288 2023-02-09 17:55:46,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:46,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:46,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4ac1d544fc33f59c1bf380e57c9b7c84, disabling compactions & flushes 2023-02-09 17:55:46,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:46,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:46,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. after waiting 0 ms 2023-02-09 17:55:46,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:46,290 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=6a8a6d1495885b8e0f6a82d61bb7547c, regionState=CLOSED 2023-02-09 17:55:46,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965346290"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965346290"}]},"ts":"1675965346290"} 2023-02-09 17:55:46,291 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5-bottom] to archive 2023-02-09 17:55:46,292 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:46,293 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-09 17:55:46,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 6a8a6d1495885b8e0f6a82d61bb7547c, server=jenkins-hbase4.apache.org,35803,1675965244552 in 177 msec 2023-02-09 17:55:46,294 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6a8a6d1495885b8e0f6a82d61bb7547c, UNASSIGN in 180 msec 2023-02-09 17:55:46,295 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,296 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,296 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,297 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,298 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,300 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3-bottom] to archive 2023-02-09 17:55:46,300 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:46,301 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,302 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,303 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,304 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,304 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,305 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,306 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:55:46,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:46,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84. 2023-02-09 17:55:46,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4ac1d544fc33f59c1bf380e57c9b7c84: Waiting for close lock at 1675965346290Running coprocessor pre-close hooks at 1675965346290Disabling compacts and flushes for region at 1675965346290Disabling writes for close at 1675965346290Writing region close event to WAL at 1675965346307 (+17 ms)Running coprocessor post-close hooks at 1675965346309 (+2 ms)Closed at 1675965346309 2023-02-09 17:55:46,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:46,311 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=4ac1d544fc33f59c1bf380e57c9b7c84, regionState=CLOSED 2023-02-09 17:55:46,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675965346311"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965346311"}]},"ts":"1675965346311"} 2023-02-09 17:55:46,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-09 17:55:46,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 4ac1d544fc33f59c1bf380e57c9b7c84, server=jenkins-hbase4.apache.org,35803,1675965244552 in 194 msec 2023-02-09 17:55:46,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-09 17:55:46,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=4ac1d544fc33f59c1bf380e57c9b7c84, UNASSIGN in 201 msec 2023-02-09 17:55:46,316 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965346316"}]},"ts":"1675965346316"} 2023-02-09 17:55:46,317 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 17:55:46,318 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-09 17:55:46,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 211 msec 2023-02-09 17:55:46,412 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,412 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 17:55:46,413 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-09 17:55:46,413 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,413 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-09 17:55:46,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,416 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,416 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 17:55:46,421 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,421 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:46,421 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:46,423 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/recovered.edits] 2023-02-09 17:55:46,423 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/recovered.edits] 2023-02-09 17:55:46,423 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/recovered.edits] 2023-02-09 17:55:46,426 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/.regioninfo to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/.regioninfo 2023-02-09 17:55:46,426 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/03fb677fe0614caebbb48fd1c6655b0d 2023-02-09 17:55:46,427 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/.regioninfo to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/.regioninfo 2023-02-09 17:55:46,428 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/295ae0a916e647448b9cae07c14de92c 2023-02-09 17:55:46,429 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/6699009e480948c9847484046a2979d5 2023-02-09 17:55:46,430 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,430 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/03fb677fe0614caebbb48fd1c6655b0d.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,431 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/92126f40b85f44a093b89234a6b6d73b 2023-02-09 17:55:46,431 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,431 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/295ae0a916e647448b9cae07c14de92c.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/c0727b31f14041b8a082b45e6a6698a1 2023-02-09 17:55:46,432 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,433 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/6699009e480948c9847484046a2979d5.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,434 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,434 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/cedebaaeeb2046e0a7e11b71ee8d899e 2023-02-09 17:55:46,434 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/92126f40b85f44a093b89234a6b6d73b.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,435 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,435 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/c0727b31f14041b8a082b45e6a6698a1.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf1/eaa421280be34203bd3d7b3b37bc2824 2023-02-09 17:55:46,437 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,437 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/cedebaaeeb2046e0a7e11b71ee8d899e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,438 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/eaa421280be34203bd3d7b3b37bc2824.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/0e4206eb1d1f456cb3240b6c0549ed0e 2023-02-09 17:55:46,440 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/19bcc43bbf5d4418a36f6512f80b74d3 2023-02-09 17:55:46,441 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,441 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/1cda9ca05a4548c5afb3f9155bad0c94 2023-02-09 17:55:46,441 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/0e4206eb1d1f456cb3240b6c0549ed0e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,442 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,442 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/6d86612bdaa44394aa833cb8f63e0454 2023-02-09 17:55:46,442 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/19bcc43bbf5d4418a36f6512f80b74d3.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,443 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,444 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/1cda9ca05a4548c5afb3f9155bad0c94.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,444 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/71ada4a5d3f24ee399db3e5068424f45 2023-02-09 17:55:46,445 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,445 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/6d86612bdaa44394aa833cb8f63e0454.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,445 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/8d4fa58b091b42ddb6dd28e21c2d640e 2023-02-09 17:55:46,446 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,446 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/71ada4a5d3f24ee399db3e5068424f45.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,447 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/cf2/b85a46f5e6e64a1a808b3bfb207137da 2023-02-09 17:55:46,447 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,448 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/8d4fa58b091b42ddb6dd28e21c2d640e.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,448 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,449 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/b85a46f5e6e64a1a808b3bfb207137da.54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,449 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/recovered.edits/85.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2/recovered.edits/85.seqid 2023-02-09 17:55:46,450 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/54ca31a1a82ef66e2bb6e7bc49e77ca2 2023-02-09 17:55:46,451 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/70c0f7e18e544adaaece127d38621f81 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf1/70c0f7e18e544adaaece127d38621f81 2023-02-09 17:55:46,451 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/2cc0a90452ea47d6bd346ff73b27632d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf1/2cc0a90452ea47d6bd346ff73b27632d 2023-02-09 17:55:46,453 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/989f7426f14e45fa89d0f78cb8dc9747 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/cf2/989f7426f14e45fa89d0f78cb8dc9747 2023-02-09 17:55:46,453 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/84ac42ac9aaf404aa1800080c301dc35 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/cf2/84ac42ac9aaf404aa1800080c301dc35 2023-02-09 17:55:46,455 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84/recovered.edits/90.seqid 2023-02-09 17:55:46,455 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c/recovered.edits/90.seqid 2023-02-09 17:55:46,456 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/4ac1d544fc33f59c1bf380e57c9b7c84 2023-02-09 17:55:46,456 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testRecoveryAndDoubleExecution/6a8a6d1495885b8e0f6a82d61bb7547c 2023-02-09 17:55:46,456 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 17:55:46,458 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,459 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-09 17:55:46,461 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-09 17:55:46,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-09 17:55:46,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965346462"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:46,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965346462"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:46,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965346462"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:46,463 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:55:46,464 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 54ca31a1a82ef66e2bb6e7bc49e77ca2, NAME => 'testRecoveryAndDoubleExecution,,1675965340530.54ca31a1a82ef66e2bb6e7bc49e77ca2.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 4ac1d544fc33f59c1bf380e57c9b7c84, NAME => 'testRecoveryAndDoubleExecution,,1675965342304.4ac1d544fc33f59c1bf380e57c9b7c84.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 6a8a6d1495885b8e0f6a82d61bb7547c, NAME => 'testRecoveryAndDoubleExecution,41,1675965342304.6a8a6d1495885b8e0f6a82d61bb7547c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 17:55:46,464 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-09 17:55:46,464 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965346464"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:46,465 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-09 17:55:46,466 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 17:55:46,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 52 msec 2023-02-09 17:55:46,517 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,517 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 17:55:46,518 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-09 17:55:46,532 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=374 (was 369) - Thread LEAK? -, OpenFileDescriptor=843 (was 745) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=243 (was 177) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=2953 (was 2986) 2023-02-09 17:55:46,546 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374, OpenFileDescriptor=843, MaxFileDescriptor=60000, SystemLoadAverage=243, ProcessCount=168, AvailableMemoryMB=2952 2023-02-09 17:55:46,546 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:46,546 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:46,546 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:46,550 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:55:46,550 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:55:46,551 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:46,552 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:55:46,553 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:55:46,554 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,555 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 empty. 2023-02-09 17:55:46,555 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,555 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 17:55:46,564 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 17:55:46,565 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 155a15a6e621abab65bad0779bd7a722, NAME => 'testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:55:46,572 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:46,572 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 155a15a6e621abab65bad0779bd7a722, disabling compactions & flushes 2023-02-09 17:55:46,572 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,572 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,572 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. after waiting 0 ms 2023-02-09 17:55:46,572 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,572 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,572 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 155a15a6e621abab65bad0779bd7a722: Waiting for close lock at 1675965346572Disabling compacts and flushes for region at 1675965346572Disabling writes for close at 1675965346572Writing region close event to WAL at 1675965346572Closed at 1675965346572 2023-02-09 17:55:46,574 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:55:46,574 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965346574"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965346574"}]},"ts":"1675965346574"} 2023-02-09 17:55:46,576 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:55:46,576 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:55:46,576 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965346576"}]},"ts":"1675965346576"} 2023-02-09 17:55:46,577 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-09 17:55:46,580 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:55:46,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:55:46,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:55:46,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:55:46,581 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:55:46,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, ASSIGN}] 2023-02-09 17:55:46,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, ASSIGN 2023-02-09 17:55:46,583 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:55:46,733 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:55:46,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=155a15a6e621abab65bad0779bd7a722, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:46,734 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965346734"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965346734"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965346734"}]},"ts":"1675965346734"} 2023-02-09 17:55:46,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 155a15a6e621abab65bad0779bd7a722, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:46,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 155a15a6e621abab65bad0779bd7a722, NAME => 'testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:55:46,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,893 INFO [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,895 DEBUG [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1 2023-02-09 17:55:46,895 DEBUG [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1 2023-02-09 17:55:46,895 INFO [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 155a15a6e621abab65bad0779bd7a722 columnFamilyName cf1 2023-02-09 17:55:46,895 INFO [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] regionserver.HStore(310): Store=155a15a6e621abab65bad0779bd7a722/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:46,895 INFO [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,896 DEBUG [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2 2023-02-09 17:55:46,896 DEBUG [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2 2023-02-09 17:55:46,897 INFO [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 155a15a6e621abab65bad0779bd7a722 columnFamilyName cf2 2023-02-09 17:55:46,897 INFO [StoreOpener-155a15a6e621abab65bad0779bd7a722-1] regionserver.HStore(310): Store=155a15a6e621abab65bad0779bd7a722/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:46,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:46,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:46,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:55:46,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 155a15a6e621abab65bad0779bd7a722; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62863448, jitterRate=-0.06326162815093994}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:46,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 155a15a6e621abab65bad0779bd7a722: Running coprocessor pre-open hook at 1675965346893Writing region info on filesystem at 1675965346893Initializing all the Stores at 1675965346893Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965346893Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965346893Cleaning up temporary data from old regions at 1675965346898 (+5 ms)Cleaning up detritus from prior splits at 1675965346898Running coprocessor post-open hooks at 1675965346901 (+3 ms)Region opened successfully at 1675965346901 2023-02-09 17:55:46,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722., pid=127, masterSystemTime=1675965346887 2023-02-09 17:55:46,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:46,904 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=155a15a6e621abab65bad0779bd7a722, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:46,904 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965346904"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965346904"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965346904"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965346904"}]},"ts":"1675965346904"} 2023-02-09 17:55:46,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-09 17:55:46,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 155a15a6e621abab65bad0779bd7a722, server=jenkins-hbase4.apache.org,43923,1675965244585 in 170 msec 2023-02-09 17:55:46,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-09 17:55:46,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, ASSIGN in 325 msec 2023-02-09 17:55:46,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:55:46,908 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965346908"}]},"ts":"1675965346908"} 2023-02-09 17:55:46,909 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-09 17:55:46,911 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:55:46,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 360 msec 2023-02-09 17:55:47,057 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,058 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,058 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:47,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,059 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,059 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,059 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 17:55:47,060 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 17:55:47,060 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,060 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,065 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,065 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,065 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,065 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,065 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,065 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,067 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:47,068 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:47,068 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,068 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. started... 2023-02-09 17:55:47,069 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:55:47,069 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,069 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,069 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,069 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,069 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,069 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,077 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/57abd863e5fb48428cadc492ed9fe68b 2023-02-09 17:55:47,086 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/5372078280f24c73a677922914c7a2f0 2023-02-09 17:55:47,089 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/57abd863e5fb48428cadc492ed9fe68b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b 2023-02-09 17:55:47,093 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:47,093 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/5372078280f24c73a677922914c7a2f0 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0 2023-02-09 17:55:47,096 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:47,097 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 28ms, sequenceid=6, compaction requested=false 2023-02-09 17:55:47,097 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,097 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 155a15a6e621abab65bad0779bd7a722: 2023-02-09 17:55:47,097 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,097 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:47,097 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:47,097 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,097 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,097 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,099 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,099 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 17:55:47,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cb1d325[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:47,101 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 17:55:47,101 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,102 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:47,102 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,102 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,106 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,106 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,106 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,106 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:47,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:55:47,109 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 17:55:47,109 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:47,109 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:47,109 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,109 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,111 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,110 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,111 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,112 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,119 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,119 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,119 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,120 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,119 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,119 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:47,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:47,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,120 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:47,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,121 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,221 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:47,221 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,221 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:47,232 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,232 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:47,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:47,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,234 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,234 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,234 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 17:55:47,234 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 17:55:47,234 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,235 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,236 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,236 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,236 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,236 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,236 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,236 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,237 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,238 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,238 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. started... 2023-02-09 17:55:47,238 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,238 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:47,239 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,240 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,241 ERROR [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6) service to localhost/127.0.0.1:38249 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:55:47,246 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/b3f6f755c896482ebf4ed77da74ea587 2023-02-09 17:55:47,656 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/c9b87c0050764915a8c72ebb117e6b5e 2023-02-09 17:55:47,659 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/b3f6f755c896482ebf4ed77da74ea587 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587 2023-02-09 17:55:47,662 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:47,663 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/c9b87c0050764915a8c72ebb117e6b5e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e 2023-02-09 17:55:47,666 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:47,667 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 429ms, sequenceid=19, compaction requested=false 2023-02-09 17:55:47,667 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 155a15a6e621abab65bad0779bd7a722: 2023-02-09 17:55:47,667 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,667 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:47,667 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:47,667 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,667 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,667 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,670 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,670 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 17:55:47,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53f07084[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:47,672 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 17:55:47,672 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,673 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,674 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,674 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,675 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:47,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:55:47,678 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 17:55:47,678 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:47,678 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:47,678 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,678 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,680 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,680 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,681 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,681 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,683 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,683 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,683 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,687 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,687 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:47,688 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,688 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,688 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,688 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,688 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:47,688 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:47,788 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:47,789 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,789 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:47,798 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,799 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:47,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:47,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,806 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,806 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,806 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 17:55:47,806 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 17:55:47,807 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,807 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,808 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,808 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,808 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,808 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,808 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,808 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:47,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,810 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,810 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,810 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,811 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,811 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,811 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,812 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,812 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,812 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:47,816 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,816 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. started... 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,816 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:47,817 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,818 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,818 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,829 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/154329083abc4903b9f24caca0d11209 2023-02-09 17:55:47,839 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 2023-02-09 17:55:47,842 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/154329083abc4903b9f24caca0d11209 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209 2023-02-09 17:55:47,845 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:47,845 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 2023-02-09 17:55:47,848 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:47,849 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 33ms, sequenceid=32, compaction requested=true 2023-02-09 17:55:47,849 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 155a15a6e621abab65bad0779bd7a722: 2023-02-09 17:55:47,849 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,849 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:47,849 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:47,849 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,849 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,849 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,852 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,852 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 17:55:47,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@291de57d[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:47,854 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 17:55:47,854 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,855 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:47,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:47,855 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:47,859 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,859 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,859 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,859 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,863 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,863 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:47,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,863 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,863 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,863 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,863 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:47,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:47,865 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:55:47,865 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 17:55:47,865 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:47,865 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:47,865 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,865 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:47,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,867 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:47,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,868 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,868 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,869 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,870 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:47,870 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,872 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,873 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,872 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,873 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,973 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:47,973 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,973 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:47,983 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:47,984 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:47,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:47,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:47,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,985 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:47,985 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,985 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 17:55:47,985 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 17:55:47,986 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,986 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,987 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,987 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,987 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,987 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,987 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:47,988 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:47,988 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,989 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,989 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:47,989 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:47,989 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:47,989 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. started... 2023-02-09 17:55:47,989 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:47,989 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:47,991 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:47,992 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:47,997 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/6eac326a8bc54a83b91939c759794a75 2023-02-09 17:55:48,006 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/d76bc8a698c44c16a5244f76981a58f9 2023-02-09 17:55:48,009 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/6eac326a8bc54a83b91939c759794a75 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75 2023-02-09 17:55:48,012 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:48,012 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/d76bc8a698c44c16a5244f76981a58f9 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9 2023-02-09 17:55:48,016 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:48,016 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 27ms, sequenceid=45, compaction requested=true 2023-02-09 17:55:48,016 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 155a15a6e621abab65bad0779bd7a722: 2023-02-09 17:55:48,016 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,016 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:48,016 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:48,016 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,016 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,016 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,018 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,018 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,018 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,018 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 17:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60564eeb[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:48,020 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 17:55:48,020 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,021 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:48,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,024 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:48,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:55:48,027 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 17:55:48,027 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:48,027 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:48,027 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,027 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:48,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,028 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,030 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,031 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,031 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,031 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,036 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:48,036 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,036 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,036 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:48,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,037 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,137 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:48,137 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:48,137 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:48,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:48,147 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:48,147 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:48,148 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:48,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:48,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,149 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:48,149 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,149 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 17:55:48,149 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 17:55:48,149 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,149 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,154 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,154 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,154 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,154 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,155 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,155 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:48,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:48,156 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:48,157 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. started... 2023-02-09 17:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,157 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,157 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,158 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,166 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/184c272f76c447039499138e1a21359a 2023-02-09 17:55:48,178 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/c9401f679a7545039f1ae63f1ab9971e 2023-02-09 17:55:48,182 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/184c272f76c447039499138e1a21359a as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a 2023-02-09 17:55:48,185 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:48,185 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/c9401f679a7545039f1ae63f1ab9971e as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e 2023-02-09 17:55:48,188 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:48,189 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 32ms, sequenceid=58, compaction requested=true 2023-02-09 17:55:48,189 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 155a15a6e621abab65bad0779bd7a722: 2023-02-09 17:55:48,189 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,189 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:48,189 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:48,189 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,189 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,189 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,191 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 17:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@223c913f[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:48,193 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 17:55:48,193 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,195 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,195 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:48,195 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:48,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,196 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,198 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:55:48,201 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 17:55:48,201 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:48,201 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:48,201 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,201 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,203 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,202 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,203 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,203 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,205 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,205 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,205 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,208 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,208 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,209 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,209 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:48,209 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,209 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,210 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,309 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:48,309 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:48,309 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:48,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:48,319 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:48,319 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:48,320 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:48,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:48,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,322 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:48,322 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,322 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 17:55:48,322 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 17:55:48,322 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,322 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,323 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,323 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,323 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,323 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,324 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,324 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,325 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,326 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,326 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,325 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,326 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. started... 2023-02-09 17:55:48,326 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:48,327 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,327 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,328 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,328 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,328 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,328 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,333 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/ff91f933175540a196d88b671ad77fef 2023-02-09 17:55:48,343 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d 2023-02-09 17:55:48,346 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/ff91f933175540a196d88b671ad77fef as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef 2023-02-09 17:55:48,349 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:48,349 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d 2023-02-09 17:55:48,353 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:48,353 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 27ms, sequenceid=71, compaction requested=true 2023-02-09 17:55:48,353 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 155a15a6e621abab65bad0779bd7a722: 2023-02-09 17:55:48,353 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,353 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:48,353 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:48,353 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,353 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 17:55:48,353 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 17:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,355 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 17:55:48,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7218b1ac[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:48,357 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 17:55:48,357 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,362 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:48,362 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:48,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 17:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,364 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,364 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,364 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,364 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,364 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 17:55:48,364 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 17:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 17:55:48,367 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 17:55:48,367 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:48,367 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:48,367 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,367 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,370 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,370 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:48,371 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:48,372 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,373 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,373 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:48,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 17:55:48,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:48,375 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:48,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,375 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:48,376 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:48,377 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,377 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,377 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 17:55:48,476 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:48,476 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:48,477 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:48,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:48,485 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,486 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=155a15a6e621abab65bad0779bd7a722, daughterA=a3a15f533c5a84f6f30c182f5668a0d8, daughterB=69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:48,487 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=155a15a6e621abab65bad0779bd7a722, daughterA=a3a15f533c5a84f6f30c182f5668a0d8, daughterB=69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:48,487 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=155a15a6e621abab65bad0779bd7a722, daughterA=a3a15f533c5a84f6f30c182f5668a0d8, daughterB=69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:48,487 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=155a15a6e621abab65bad0779bd7a722, daughterA=a3a15f533c5a84f6f30c182f5668a0d8, daughterB=69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:48,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, UNASSIGN}] 2023-02-09 17:55:48,489 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, UNASSIGN 2023-02-09 17:55:48,490 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=155a15a6e621abab65bad0779bd7a722, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:48,490 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965348490"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965348490"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965348490"}]},"ts":"1675965348490"} 2023-02-09 17:55:48,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 155a15a6e621abab65bad0779bd7a722, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:48,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 155a15a6e621abab65bad0779bd7a722, disabling compactions & flushes 2023-02-09 17:55:48,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. after waiting 0 ms 2023-02-09 17:55:48,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 155a15a6e621abab65bad0779bd7a722 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:55:48,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/686b3287c5734e748d998ce8034b484b 2023-02-09 17:55:48,663 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/a10dc11117be4eb69ca4295d0afec08b 2023-02-09 17:55:48,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf1/686b3287c5734e748d998ce8034b484b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b 2023-02-09 17:55:48,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:48,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.tmp/cf2/a10dc11117be4eb69ca4295d0afec08b as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b 2023-02-09 17:55:48,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:48,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 30ms, sequenceid=82, compaction requested=true 2023-02-09 17:55:48,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:55:48,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:48,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. 2023-02-09 17:55:48,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 155a15a6e621abab65bad0779bd7a722: Waiting for close lock at 1675965348643Running coprocessor pre-close hooks at 1675965348643Disabling compacts and flushes for region at 1675965348643Disabling writes for close at 1675965348643Obtaining lock to block concurrent updates at 1675965348643Preparing flush snapshotting stores in 155a15a6e621abab65bad0779bd7a722 at 1675965348643Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675965348643Flushing stores of testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. at 1675965348644 (+1 ms)Flushing 155a15a6e621abab65bad0779bd7a722/cf1: creating writer at 1675965348646 (+2 ms)Flushing 155a15a6e621abab65bad0779bd7a722/cf1: appending metadata at 1675965348648 (+2 ms)Flushing 155a15a6e621abab65bad0779bd7a722/cf1: closing flushed file at 1675965348648Flushing 155a15a6e621abab65bad0779bd7a722/cf2: creating writer at 1675965348656 (+8 ms)Flushing 155a15a6e621abab65bad0779bd7a722/cf2: appending metadata at 1675965348658 (+2 ms)Flushing 155a15a6e621abab65bad0779bd7a722/cf2: closing flushed file at 1675965348658Flushing 155a15a6e621abab65bad0779bd7a722/cf1: reopening flushed file at 1675965348667 (+9 ms)Flushing 155a15a6e621abab65bad0779bd7a722/cf2: reopening flushed file at 1675965348670 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 155a15a6e621abab65bad0779bd7a722 in 30ms, sequenceid=82, compaction requested=true at 1675965348673 (+3 ms)Writing region close event to WAL at 1675965348676 (+3 ms)Running coprocessor post-close hooks at 1675965348678 (+2 ms)Closed at 1675965348678 2023-02-09 17:55:48,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=155a15a6e621abab65bad0779bd7a722, regionState=CLOSED 2023-02-09 17:55:48,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965348680"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965348680"}]},"ts":"1675965348680"} 2023-02-09 17:55:48,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-09 17:55:48,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 155a15a6e621abab65bad0779bd7a722, server=jenkins-hbase4.apache.org,43923,1675965244585 in 191 msec 2023-02-09 17:55:48,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-09 17:55:48,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=155a15a6e621abab65bad0779bd7a722, UNASSIGN in 194 msec 2023-02-09 17:55:48,695 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=155a15a6e621abab65bad0779bd7a722, threads=14 2023-02-09 17:55:48,696 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,696 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,696 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,696 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,696 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,700 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,700 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,700 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,704 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,704 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,704 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,704 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,705 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,707 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,711 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e, top=false 2023-02-09 17:55:48,711 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9, top=false 2023-02-09 17:55:48,711 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0, top=false 2023-02-09 17:55:48,711 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef, top=false 2023-02-09 17:55:48,713 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a, top=false 2023-02-09 17:55:48,723 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e, top=false 2023-02-09 17:55:48,724 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d, top=false 2023-02-09 17:55:48,735 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9 for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,735 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,736 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b, top=false 2023-02-09 17:55:48,737 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,737 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6, top=false 2023-02-09 17:55:48,737 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75, top=false 2023-02-09 17:55:48,738 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209, top=false 2023-02-09 17:55:48,752 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,752 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,752 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587, top=false 2023-02-09 17:55:48,753 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0 for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,753 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b, top=false 2023-02-09 17:55:48,754 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b, top=false 2023-02-09 17:55:48,754 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,755 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,755 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,755 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,759 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209 for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,760 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6 for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,761 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,762 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,762 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,763 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,764 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,766 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,769 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,769 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,779 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587 for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,779 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75 for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,784 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,784 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75 for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,788 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,792 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/.splits/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b for child: a3a15f533c5a84f6f30c182f5668a0d8, parent: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,793 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,796 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:55:48,799 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b for region: 155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:48,799 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 155a15a6e621abab65bad0779bd7a722 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-09 17:55:48,864 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:55:48,866 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:55:48,868 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675965348868"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675965348868"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675965348868"}]},"ts":"1675965348868"} 2023-02-09 17:55:48,868 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965348868"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965348868"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965348868"}]},"ts":"1675965348868"} 2023-02-09 17:55:48,868 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965348868"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965348868"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965348868"}]},"ts":"1675965348868"} 2023-02-09 17:55:48,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, ASSIGN}] 2023-02-09 17:55:48,872 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, ASSIGN 2023-02-09 17:55:48,873 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:55:48,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, ASSIGN 2023-02-09 17:55:48,874 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:55:49,023 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:55:49,025 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a3a15f533c5a84f6f30c182f5668a0d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,025 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349025"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965349025"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965349025"}]},"ts":"1675965349025"} 2023-02-09 17:55:49,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure a3a15f533c5a84f6f30c182f5668a0d8, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:49,029 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=69fb2ea3f1aae9e34b41bb92546bba60, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,029 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349029"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965349029"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965349029"}]},"ts":"1675965349029"} 2023-02-09 17:55:49,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 69fb2ea3f1aae9e34b41bb92546bba60, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:49,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 69fb2ea3f1aae9e34b41bb92546bba60, NAME => 'testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.', STARTKEY => '71', ENDKEY => ''} 2023-02-09 17:55:49,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:49,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:49,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,182 INFO [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,183 DEBUG [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/cf1 2023-02-09 17:55:49,183 DEBUG [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/cf1 2023-02-09 17:55:49,184 INFO [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 69fb2ea3f1aae9e34b41bb92546bba60 columnFamilyName cf1 2023-02-09 17:55:49,184 INFO [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] regionserver.HStore(310): Store=69fb2ea3f1aae9e34b41bb92546bba60/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:49,184 INFO [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,185 DEBUG [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/cf2 2023-02-09 17:55:49,186 DEBUG [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/cf2 2023-02-09 17:55:49,186 INFO [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 69fb2ea3f1aae9e34b41bb92546bba60 columnFamilyName cf2 2023-02-09 17:55:49,186 INFO [StoreOpener-69fb2ea3f1aae9e34b41bb92546bba60-1] regionserver.HStore(310): Store=69fb2ea3f1aae9e34b41bb92546bba60/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:49,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:49,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 69fb2ea3f1aae9e34b41bb92546bba60; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72100230, jitterRate=0.0743771493434906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:49,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 69fb2ea3f1aae9e34b41bb92546bba60: Running coprocessor pre-open hook at 1675965349181Writing region info on filesystem at 1675965349181Initializing all the Stores at 1675965349182 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965349182Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965349182Cleaning up temporary data from old regions at 1675965349188 (+6 ms)Cleaning up detritus from prior splits at 1675965349189 (+1 ms)Running coprocessor post-open hooks at 1675965349191 (+2 ms)Region opened successfully at 1675965349191 2023-02-09 17:55:49,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60., pid=134, masterSystemTime=1675965349178 2023-02-09 17:55:49,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3a15f533c5a84f6f30c182f5668a0d8, NAME => 'testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.', STARTKEY => '', ENDKEY => '71'} 2023-02-09 17:55:49,194 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=69fb2ea3f1aae9e34b41bb92546bba60, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:49,194 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349193"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965349193"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965349193"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965349193"}]},"ts":"1675965349193"} 2023-02-09 17:55:49,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:49,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,195 INFO [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-09 17:55:49,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 69fb2ea3f1aae9e34b41bb92546bba60, server=jenkins-hbase4.apache.org,43923,1675965244585 in 165 msec 2023-02-09 17:55:49,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, ASSIGN in 325 msec 2023-02-09 17:55:49,200 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1 2023-02-09 17:55:49,200 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1 2023-02-09 17:55:49,200 INFO [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3a15f533c5a84f6f30c182f5668a0d8 columnFamilyName cf1 2023-02-09 17:55:49,205 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209 2023-02-09 17:55:49,207 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a 2023-02-09 17:55:49,210 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b 2023-02-09 17:55:49,212 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b 2023-02-09 17:55:49,214 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75 2023-02-09 17:55:49,217 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587 2023-02-09 17:55:49,219 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef 2023-02-09 17:55:49,219 INFO [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(310): Store=a3a15f533c5a84f6f30c182f5668a0d8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:49,219 INFO [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,220 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2 2023-02-09 17:55:49,220 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2 2023-02-09 17:55:49,220 INFO [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3a15f533c5a84f6f30c182f5668a0d8 columnFamilyName cf2 2023-02-09 17:55:49,224 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d 2023-02-09 17:55:49,226 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6 2023-02-09 17:55:49,228 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0 2023-02-09 17:55:49,231 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b 2023-02-09 17:55:49,233 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e 2023-02-09 17:55:49,236 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e 2023-02-09 17:55:49,239 DEBUG [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9 2023-02-09 17:55:49,239 INFO [StoreOpener-a3a15f533c5a84f6f30c182f5668a0d8-1] regionserver.HStore(310): Store=a3a15f533c5a84f6f30c182f5668a0d8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:49,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:49,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a3a15f533c5a84f6f30c182f5668a0d8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67607914, jitterRate=0.007436424493789673}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:49,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a3a15f533c5a84f6f30c182f5668a0d8: Running coprocessor pre-open hook at 1675965349194Writing region info on filesystem at 1675965349194Initializing all the Stores at 1675965349195 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965349195Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965349195Cleaning up temporary data from old regions at 1675965349241 (+46 ms)Cleaning up detritus from prior splits at 1675965349241Running coprocessor post-open hooks at 1675965349243 (+2 ms)Region opened successfully at 1675965349243 2023-02-09 17:55:49,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8., pid=133, masterSystemTime=1675965349178 2023-02-09 17:55:49,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:49,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:49,245 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:49,245 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:49,245 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 17:55:49,245 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 17:55:49,245 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1912): a3a15f533c5a84f6f30c182f5668a0d8/cf2 is initiating minor compaction (all files) 2023-02-09 17:55:49,245 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1912): a3a15f533c5a84f6f30c182f5668a0d8/cf1 is initiating minor compaction (all files) 2023-02-09 17:55:49,246 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a3a15f533c5a84f6f30c182f5668a0d8/cf2 in testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,246 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a3a15f533c5a84f6f30c182f5668a0d8/cf1 in testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,246 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/.tmp, totalSize=34.9 K 2023-02-09 17:55:49,246 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/.tmp, totalSize=34.9 K 2023-02-09 17:55:49,246 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965347054 2023-02-09 17:55:49,246 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965347054 2023-02-09 17:55:49,246 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965347222 2023-02-09 17:55:49,246 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965347222 2023-02-09 17:55:49,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,247 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965347790 2023-02-09 17:55:49,247 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965347790 2023-02-09 17:55:49,247 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=a3a15f533c5a84f6f30c182f5668a0d8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,247 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965347975 2023-02-09 17:55:49,247 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349247"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965349247"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965349247"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965349247"}]},"ts":"1675965349247"} 2023-02-09 17:55:49,247 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965347975 2023-02-09 17:55:49,247 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965348139 2023-02-09 17:55:49,248 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965348139 2023-02-09 17:55:49,248 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965348311 2023-02-09 17:55:49,248 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965348311 2023-02-09 17:55:49,248 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965348478 2023-02-09 17:55:49,248 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965348478 2023-02-09 17:55:49,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-09 17:55:49,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure a3a15f533c5a84f6f30c182f5668a0d8, server=jenkins-hbase4.apache.org,43923,1675965244585 in 223 msec 2023-02-09 17:55:49,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-09 17:55:49,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, ASSIGN in 384 msec 2023-02-09 17:55:49,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=155a15a6e621abab65bad0779bd7a722, daughterA=a3a15f533c5a84f6f30c182f5668a0d8, daughterB=69fb2ea3f1aae9e34b41bb92546bba60 in 771 msec 2023-02-09 17:55:49,266 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] throttle.PressureAwareThroughputController(145): a3a15f533c5a84f6f30c182f5668a0d8#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:49,266 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] throttle.PressureAwareThroughputController(145): a3a15f533c5a84f6f30c182f5668a0d8#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:49,280 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/.tmp/cf1/31a1ec19455e4b6eb59ff1cb14463806 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/31a1ec19455e4b6eb59ff1cb14463806 2023-02-09 17:55:49,280 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/.tmp/cf2/161b617e02d14799ada380b78a7c4bb6 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/161b617e02d14799ada380b78a7c4bb6 2023-02-09 17:55:49,284 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a3a15f533c5a84f6f30c182f5668a0d8/cf2 of a3a15f533c5a84f6f30c182f5668a0d8 into 161b617e02d14799ada380b78a7c4bb6(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:49,284 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a3a15f533c5a84f6f30c182f5668a0d8/cf1 of a3a15f533c5a84f6f30c182f5668a0d8 into 31a1ec19455e4b6eb59ff1cb14463806(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:49,284 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a3a15f533c5a84f6f30c182f5668a0d8: 2023-02-09 17:55:49,284 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a3a15f533c5a84f6f30c182f5668a0d8: 2023-02-09 17:55:49,284 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8., storeName=a3a15f533c5a84f6f30c182f5668a0d8/cf2, priority=9, startTime=1675965349245; duration=0sec 2023-02-09 17:55:49,284 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8., storeName=a3a15f533c5a84f6f30c182f5668a0d8/cf1, priority=9, startTime=1675965349244; duration=0sec 2023-02-09 17:55:49,284 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:49,285 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:49,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43923] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40196 deadline: 1675965409487, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722. is not online on jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,597 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-09 17:55:49,600 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:49,600 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,600 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,602 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,602 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,602 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,604 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,604 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 17:55:49,605 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965349605"}]},"ts":"1675965349605"} 2023-02-09 17:55:49,606 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-09 17:55:49,608 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-09 17:55:49,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, UNASSIGN}] 2023-02-09 17:55:49,610 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, UNASSIGN 2023-02-09 17:55:49,610 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=69fb2ea3f1aae9e34b41bb92546bba60, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,610 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349610"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965349610"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965349610"}]},"ts":"1675965349610"} 2023-02-09 17:55:49,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 69fb2ea3f1aae9e34b41bb92546bba60, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:49,613 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, UNASSIGN 2023-02-09 17:55:49,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a3a15f533c5a84f6f30c182f5668a0d8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:49,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349614"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965349614"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965349614"}]},"ts":"1675965349614"} 2023-02-09 17:55:49,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure a3a15f533c5a84f6f30c182f5668a0d8, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:49,705 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,705 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 17:55:49,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 69fb2ea3f1aae9e34b41bb92546bba60, disabling compactions & flushes 2023-02-09 17:55:49,764 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. after waiting 0 ms 2023-02-09 17:55:49,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-09 17:55:49,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:49,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60. 2023-02-09 17:55:49,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 69fb2ea3f1aae9e34b41bb92546bba60: Waiting for close lock at 1675965349763Running coprocessor pre-close hooks at 1675965349763Disabling compacts and flushes for region at 1675965349763Disabling writes for close at 1675965349764 (+1 ms)Writing region close event to WAL at 1675965349764Running coprocessor post-close hooks at 1675965349767 (+3 ms)Closed at 1675965349767 2023-02-09 17:55:49,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a3a15f533c5a84f6f30c182f5668a0d8, disabling compactions & flushes 2023-02-09 17:55:49,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. after waiting 0 ms 2023-02-09 17:55:49,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,769 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=69fb2ea3f1aae9e34b41bb92546bba60, regionState=CLOSED 2023-02-09 17:55:49,769 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349769"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965349769"}]},"ts":"1675965349769"} 2023-02-09 17:55:49,769 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b] to archive 2023-02-09 17:55:49,770 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:49,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-09 17:55:49,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 69fb2ea3f1aae9e34b41bb92546bba60, server=jenkins-hbase4.apache.org,43923,1675965244585 in 159 msec 2023-02-09 17:55:49,772 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-57abd863e5fb48428cadc492ed9fe68b 2023-02-09 17:55:49,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=69fb2ea3f1aae9e34b41bb92546bba60, UNASSIGN in 163 msec 2023-02-09 17:55:49,773 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-b3f6f755c896482ebf4ed77da74ea587 2023-02-09 17:55:49,774 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-154329083abc4903b9f24caca0d11209 2023-02-09 17:55:49,775 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-6eac326a8bc54a83b91939c759794a75 2023-02-09 17:55:49,776 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-184c272f76c447039499138e1a21359a 2023-02-09 17:55:49,776 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-ff91f933175540a196d88b671ad77fef 2023-02-09 17:55:49,777 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-686b3287c5734e748d998ce8034b484b 2023-02-09 17:55:49,778 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b] to archive 2023-02-09 17:55:49,779 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:49,780 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-5372078280f24c73a677922914c7a2f0 2023-02-09 17:55:49,781 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9b87c0050764915a8c72ebb117e6b5e 2023-02-09 17:55:49,782 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-2041792d9c394fe0b28a7ac45fd6d4c6 2023-02-09 17:55:49,782 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-d76bc8a698c44c16a5244f76981a58f9 2023-02-09 17:55:49,783 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-c9401f679a7545039f1ae63f1ab9971e 2023-02-09 17:55:49,784 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-1a8488d1e39d4e2ca0187a4058aa0b0d 2023-02-09 17:55:49,785 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/testSplitTableRegionEmptyDaughter=155a15a6e621abab65bad0779bd7a722-a10dc11117be4eb69ca4295d0afec08b 2023-02-09 17:55:49,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:55:49,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:49,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8. 2023-02-09 17:55:49,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a3a15f533c5a84f6f30c182f5668a0d8: Waiting for close lock at 1675965349769Running coprocessor pre-close hooks at 1675965349769Disabling compacts and flushes for region at 1675965349769Disabling writes for close at 1675965349769Writing region close event to WAL at 1675965349785 (+16 ms)Running coprocessor post-close hooks at 1675965349787 (+2 ms)Closed at 1675965349788 (+1 ms) 2023-02-09 17:55:49,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,789 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=a3a15f533c5a84f6f30c182f5668a0d8, regionState=CLOSED 2023-02-09 17:55:49,789 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675965349789"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965349789"}]},"ts":"1675965349789"} 2023-02-09 17:55:49,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-09 17:55:49,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure a3a15f533c5a84f6f30c182f5668a0d8, server=jenkins-hbase4.apache.org,43923,1675965244585 in 175 msec 2023-02-09 17:55:49,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-09 17:55:49,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a3a15f533c5a84f6f30c182f5668a0d8, UNASSIGN in 182 msec 2023-02-09 17:55:49,793 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965349792"}]},"ts":"1675965349792"} 2023-02-09 17:55:49,793 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-09 17:55:49,795 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-09 17:55:49,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 193 msec 2023-02-09 17:55:49,906 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,907 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 17:55:49,908 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-09 17:55:49,908 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,908 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,911 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:49,911 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:49,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 17:55:49,915 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:49,915 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,915 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,917 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/recovered.edits] 2023-02-09 17:55:49,917 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/recovered.edits] 2023-02-09 17:55:49,918 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/recovered.edits] 2023-02-09 17:55:49,920 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/154329083abc4903b9f24caca0d11209 2023-02-09 17:55:49,921 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/31a1ec19455e4b6eb59ff1cb14463806 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf1/31a1ec19455e4b6eb59ff1cb14463806 2023-02-09 17:55:49,921 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/184c272f76c447039499138e1a21359a 2023-02-09 17:55:49,922 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/recovered.edits/88.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60/recovered.edits/88.seqid 2023-02-09 17:55:49,922 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/57abd863e5fb48428cadc492ed9fe68b 2023-02-09 17:55:49,923 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/69fb2ea3f1aae9e34b41bb92546bba60 2023-02-09 17:55:49,923 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/161b617e02d14799ada380b78a7c4bb6 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/cf2/161b617e02d14799ada380b78a7c4bb6 2023-02-09 17:55:49,924 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/686b3287c5734e748d998ce8034b484b 2023-02-09 17:55:49,925 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/6eac326a8bc54a83b91939c759794a75 2023-02-09 17:55:49,925 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8/recovered.edits/90.seqid 2023-02-09 17:55:49,926 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/b3f6f755c896482ebf4ed77da74ea587 2023-02-09 17:55:49,926 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/a3a15f533c5a84f6f30c182f5668a0d8 2023-02-09 17:55:49,926 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf1/ff91f933175540a196d88b671ad77fef 2023-02-09 17:55:49,928 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/1a8488d1e39d4e2ca0187a4058aa0b0d 2023-02-09 17:55:49,929 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/2041792d9c394fe0b28a7ac45fd6d4c6 2023-02-09 17:55:49,930 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/5372078280f24c73a677922914c7a2f0 2023-02-09 17:55:49,931 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/a10dc11117be4eb69ca4295d0afec08b 2023-02-09 17:55:49,932 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9401f679a7545039f1ae63f1ab9971e 2023-02-09 17:55:49,933 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/c9b87c0050764915a8c72ebb117e6b5e 2023-02-09 17:55:49,934 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/cf2/d76bc8a698c44c16a5244f76981a58f9 2023-02-09 17:55:49,935 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/recovered.edits/85.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722/recovered.edits/85.seqid 2023-02-09 17:55:49,936 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegionEmptyDaughter/155a15a6e621abab65bad0779bd7a722 2023-02-09 17:55:49,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 17:55:49,937 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,939 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-09 17:55:49,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-09 17:55:49,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-09 17:55:49,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965349941"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:49,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965349941"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:49,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965349941"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:49,943 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:55:49,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 155a15a6e621abab65bad0779bd7a722, NAME => 'testSplitTableRegionEmptyDaughter,,1675965346550.155a15a6e621abab65bad0779bd7a722.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a3a15f533c5a84f6f30c182f5668a0d8, NAME => 'testSplitTableRegionEmptyDaughter,,1675965348485.a3a15f533c5a84f6f30c182f5668a0d8.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 69fb2ea3f1aae9e34b41bb92546bba60, NAME => 'testSplitTableRegionEmptyDaughter,71,1675965348485.69fb2ea3f1aae9e34b41bb92546bba60.', STARTKEY => '71', ENDKEY => ''}] 2023-02-09 17:55:49,943 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-09 17:55:49,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965349943"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:49,944 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-09 17:55:49,946 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 17:55:49,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 38 msec 2023-02-09 17:55:50,012 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,012 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 17:55:50,013 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-09 17:55:50,027 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 374) - Thread LEAK? -, OpenFileDescriptor=845 (was 843) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=223 (was 243), ProcessCount=168 (was 168), AvailableMemoryMB=2949 (was 2952) 2023-02-09 17:55:50,040 INFO [Listener at localhost/34581] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=845, MaxFileDescriptor=60000, SystemLoadAverage=223, ProcessCount=168, AvailableMemoryMB=2949 2023-02-09 17:55:50,040 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:50,040 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,040 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,043 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 17:55:50,043 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 17:55:50,044 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-09 17:55:50,045 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 17:55:50,046 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 17:55:50,047 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,047 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 empty. 2023-02-09 17:55:50,048 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,048 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 17:55:50,056 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-09 17:55:50,058 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9811a03b0c4db1253c79047c0b600442, NAME => 'testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp 2023-02-09 17:55:50,064 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:50,064 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 9811a03b0c4db1253c79047c0b600442, disabling compactions & flushes 2023-02-09 17:55:50,064 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,064 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,064 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. after waiting 0 ms 2023-02-09 17:55:50,064 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,064 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,064 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 9811a03b0c4db1253c79047c0b600442: Waiting for close lock at 1675965350064Disabling compacts and flushes for region at 1675965350064Disabling writes for close at 1675965350064Writing region close event to WAL at 1675965350064Closed at 1675965350064 2023-02-09 17:55:50,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 17:55:50,066 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965350066"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965350066"}]},"ts":"1675965350066"} 2023-02-09 17:55:50,067 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 17:55:50,068 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 17:55:50,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965350068"}]},"ts":"1675965350068"} 2023-02-09 17:55:50,069 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-09 17:55:50,072 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 17:55:50,073 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 17:55:50,073 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 17:55:50,073 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 17:55:50,073 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 17:55:50,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, ASSIGN}] 2023-02-09 17:55:50,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, ASSIGN 2023-02-09 17:55:50,074 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:55:50,225 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 17:55:50,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=9811a03b0c4db1253c79047c0b600442, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965350226"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965350226"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965350226"}]},"ts":"1675965350226"} 2023-02-09 17:55:50,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 9811a03b0c4db1253c79047c0b600442, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:50,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9811a03b0c4db1253c79047c0b600442, NAME => 'testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.', STARTKEY => '', ENDKEY => ''} 2023-02-09 17:55:50,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:50,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:50,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,382 INFO [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,383 DEBUG [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1 2023-02-09 17:55:50,383 DEBUG [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1 2023-02-09 17:55:50,383 INFO [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9811a03b0c4db1253c79047c0b600442 columnFamilyName cf1 2023-02-09 17:55:50,384 INFO [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] regionserver.HStore(310): Store=9811a03b0c4db1253c79047c0b600442/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:50,384 INFO [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,385 DEBUG [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2 2023-02-09 17:55:50,385 DEBUG [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2 2023-02-09 17:55:50,385 INFO [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9811a03b0c4db1253c79047c0b600442 columnFamilyName cf2 2023-02-09 17:55:50,386 INFO [StoreOpener-9811a03b0c4db1253c79047c0b600442-1] regionserver.HStore(310): Store=9811a03b0c4db1253c79047c0b600442/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:50,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:50,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:50,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 17:55:50,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9811a03b0c4db1253c79047c0b600442; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68610485, jitterRate=0.02237589657306671}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:50,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9811a03b0c4db1253c79047c0b600442: Running coprocessor pre-open hook at 1675965350381Writing region info on filesystem at 1675965350381Initializing all the Stores at 1675965350382 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965350382Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965350382Cleaning up temporary data from old regions at 1675965350387 (+5 ms)Cleaning up detritus from prior splits at 1675965350387Running coprocessor post-open hooks at 1675965350390 (+3 ms)Region opened successfully at 1675965350390 2023-02-09 17:55:50,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442., pid=143, masterSystemTime=1675965350378 2023-02-09 17:55:50,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,392 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=9811a03b0c4db1253c79047c0b600442, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,392 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965350392"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965350392"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965350392"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965350392"}]},"ts":"1675965350392"} 2023-02-09 17:55:50,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-09 17:55:50,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 9811a03b0c4db1253c79047c0b600442, server=jenkins-hbase4.apache.org,43923,1675965244585 in 166 msec 2023-02-09 17:55:50,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-09 17:55:50,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, ASSIGN in 321 msec 2023-02-09 17:55:50,396 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 17:55:50,396 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965350396"}]},"ts":"1675965350396"} 2023-02-09 17:55:50,397 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-09 17:55:50,399 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 17:55:50,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 356 msec 2023-02-09 17:55:50,551 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,551 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,551 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:50,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 17:55:50,553 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 17:55:50,553 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,553 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 17:55:50,553 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 17:55:50,554 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,554 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,555 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,555 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,555 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,555 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,555 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,555 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,556 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:50,557 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:50,557 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,557 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. started... 2023-02-09 17:55:50,558 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 17:55:50,559 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,559 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,560 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,569 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/f7e6f76588a34d5297df48ad59282372 2023-02-09 17:55:50,590 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/2aaa963af97b496c95a23216cd0ecabb 2023-02-09 17:55:50,593 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/f7e6f76588a34d5297df48ad59282372 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372 2023-02-09 17:55:50,596 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:50,596 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/2aaa963af97b496c95a23216cd0ecabb as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb 2023-02-09 17:55:50,599 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:50,599 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 41ms, sequenceid=6, compaction requested=false 2023-02-09 17:55:50,599 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-09 17:55:50,600 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9811a03b0c4db1253c79047c0b600442: 2023-02-09 17:55:50,600 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,600 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:50,600 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:50,600 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,600 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,600 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,601 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,601 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,602 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,602 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 17:55:50,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34e8849f[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:50,603 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 17:55:50,603 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,606 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,606 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:50,606 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:50,607 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,607 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,607 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,607 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,611 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,611 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,611 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,611 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,611 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,611 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:50,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 17:55:50,613 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 17:55:50,613 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:50,613 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:50,613 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 17:55:50,613 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:50,615 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,615 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,615 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,615 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,615 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,616 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,616 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,615 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,616 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,616 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,619 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,620 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,620 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,622 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,622 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,623 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,622 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,623 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 17:55:50,623 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:50,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:50,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,623 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,624 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,724 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:50,724 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,724 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:50,733 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,733 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,734 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:50,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:50,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 17:55:50,735 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 17:55:50,735 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,735 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 17:55:50,735 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 17:55:50,736 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,736 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,737 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,737 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,737 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,737 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:50,738 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,738 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,738 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,738 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:50,739 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:50,739 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,739 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. started... 2023-02-09 17:55:50,739 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:50,740 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,742 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,742 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,742 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,742 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,748 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/223bfea8f56c47dd994490e33f0ccdaf 2023-02-09 17:55:50,758 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/b74a4a29d13c4af1908ace075eda8226 2023-02-09 17:55:50,761 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/223bfea8f56c47dd994490e33f0ccdaf as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf 2023-02-09 17:55:50,764 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:50,765 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/b74a4a29d13c4af1908ace075eda8226 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226 2023-02-09 17:55:50,768 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 17:55:50,768 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 29ms, sequenceid=19, compaction requested=false 2023-02-09 17:55:50,768 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9811a03b0c4db1253c79047c0b600442: 2023-02-09 17:55:50,769 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:50,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:50,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,769 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,770 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,770 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,770 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,770 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 17:55:50,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@661ba5ff[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:50,772 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 17:55:50,772 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,773 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,773 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,773 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:50,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:50,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,774 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,774 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,774 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:50,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,774 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,774 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,774 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:50,775 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,775 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,775 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,775 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,776 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,776 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,776 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,777 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,777 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,777 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:50,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 17:55:50,779 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 17:55:50,780 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:50,780 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:50,780 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 17:55:50,780 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,782 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,781 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,782 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,782 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,784 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,784 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,784 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,786 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,786 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,786 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,786 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,787 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,786 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 17:55:50,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:50,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,787 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,788 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,888 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:50,888 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,888 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:50,898 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:50,898 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:50,898 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:50,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:50,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 17:55:50,900 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 17:55:50,900 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,900 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 17:55:50,900 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 17:55:50,900 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,900 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,903 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,903 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,903 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,903 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,903 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,903 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:50,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,904 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:50,904 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:50,905 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,905 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,905 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,905 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,905 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:50,905 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:50,905 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,905 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. started... 2023-02-09 17:55:50,906 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:50,909 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,909 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,909 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,909 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,909 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,909 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,919 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/c5ad951842c14b85bedfc01bdf554428 2023-02-09 17:55:50,929 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 2023-02-09 17:55:50,932 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/c5ad951842c14b85bedfc01bdf554428 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428 2023-02-09 17:55:50,935 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:50,936 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 2023-02-09 17:55:50,939 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 17:55:50,940 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 34ms, sequenceid=32, compaction requested=true 2023-02-09 17:55:50,940 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9811a03b0c4db1253c79047c0b600442: 2023-02-09 17:55:50,940 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:50,940 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:50,942 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,942 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,942 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,942 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 17:55:50,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2068756a[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:50,944 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 17:55:50,944 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,945 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,945 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,945 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,946 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,946 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,946 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,946 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,946 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:50,946 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,946 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:50,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,946 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:50,949 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,950 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,950 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,950 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,952 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:50,952 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,952 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,952 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:50,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 17:55:50,955 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 17:55:50,955 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:50,955 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:50,955 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 17:55:50,955 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:50,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,958 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:50,959 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:50,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:50,960 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,960 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,961 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:50,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 17:55:50,964 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:50,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,964 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:50,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:50,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:50,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:50,964 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:50,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:50,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:50,965 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:50,966 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,065 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:51,066 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,066 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:51,075 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,076 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:51,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:51,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 17:55:51,078 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 17:55:51,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 17:55:51,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 17:55:51,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,078 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,079 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,079 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,079 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,079 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,080 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,080 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:51,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:51,082 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,081 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,082 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. started... 2023-02-09 17:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,082 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,082 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,083 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,084 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,084 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,085 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,085 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,085 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,089 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/f433b8e508094de5bba3cc95478368ed 2023-02-09 17:55:51,099 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/3e5969927a8d4cda8a7161fc376aecd2 2023-02-09 17:55:51,102 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/f433b8e508094de5bba3cc95478368ed as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed 2023-02-09 17:55:51,105 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:51,105 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/3e5969927a8d4cda8a7161fc376aecd2 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2 2023-02-09 17:55:51,108 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 17:55:51,109 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 27ms, sequenceid=45, compaction requested=true 2023-02-09 17:55:51,109 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9811a03b0c4db1253c79047c0b600442: 2023-02-09 17:55:51,109 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,109 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,111 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,111 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,112 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,112 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 17:55:51,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@174a251a[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:51,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 17:55:51,113 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,115 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,115 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:51,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,115 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,117 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,118 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,118 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:51,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 17:55:51,121 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 17:55:51,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:51,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:51,121 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 17:55:51,122 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,126 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,127 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,127 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,127 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,129 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,129 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,130 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,136 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,136 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 17:55:51,137 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:51,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:51,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:51,136 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,137 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,138 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,138 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,237 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:51,237 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,237 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:51,248 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,248 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,248 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:51,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:51,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 17:55:51,250 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 17:55:51,250 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,250 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 17:55:51,250 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 17:55:51,250 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,250 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,252 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,252 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,252 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,252 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:51,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,253 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,253 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,254 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,254 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. started... 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,254 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,256 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,257 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,257 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,257 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,265 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/bc6a2b04b32546c191f68ade9b454253 2023-02-09 17:55:51,277 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/e74297e27be342c4b1cc9908a25c3cfb 2023-02-09 17:55:51,280 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/bc6a2b04b32546c191f68ade9b454253 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253 2023-02-09 17:55:51,283 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:51,283 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/e74297e27be342c4b1cc9908a25c3cfb as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb 2023-02-09 17:55:51,286 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 17:55:51,287 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 33ms, sequenceid=58, compaction requested=true 2023-02-09 17:55:51,287 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9811a03b0c4db1253c79047c0b600442: 2023-02-09 17:55:51,287 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,288 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,288 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,289 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,289 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 17:55:51,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67769710[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:51,290 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 17:55:51,290 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,291 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:51,291 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,292 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:51,293 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,293 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,293 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,293 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,293 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,293 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,295 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,295 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,295 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,295 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:51,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 17:55:51,298 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 17:55:51,298 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:51,298 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:51,298 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 17:55:51,298 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:51,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,299 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,300 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,300 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,301 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,302 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,302 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,302 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,305 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,304 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 17:55:51,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:51,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:51,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,305 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,306 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,406 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:51,406 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,406 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:51,419 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,419 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,420 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 17:55:51,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 17:55:51,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 17:55:51,421 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 17:55:51,421 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,421 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 17:55:51,421 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 17:55:51,422 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,422 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,423 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,423 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,423 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,423 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,423 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,423 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 17:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 17:55:51,424 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,424 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,44623,1675965244508' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 17:55:51,425 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,425 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35803,1675965244552' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,425 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. started... 2023-02-09 17:55:51,425 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 17:55:51,428 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,428 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,428 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,428 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,429 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,429 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,433 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/4487f45ff20f4e94a59543427db1eb27 2023-02-09 17:55:51,442 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/772b386070354d12b4e3b93b2cb081e7 2023-02-09 17:55:51,445 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/4487f45ff20f4e94a59543427db1eb27 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27 2023-02-09 17:55:51,448 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:51,449 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/772b386070354d12b4e3b93b2cb081e7 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7 2023-02-09 17:55:51,451 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 17:55:51,452 INFO [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 27ms, sequenceid=71, compaction requested=true 2023-02-09 17:55:51,452 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9811a03b0c4db1253c79047c0b600442: 2023-02-09 17:55:51,452 DEBUG [rs(jenkins-hbase4.apache.org,43923,1675965244585)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,452 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 17:55:51,452 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 17:55:51,452 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,452 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 17:55:51,452 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 17:55:51,453 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,453 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43923,1675965244585' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 17:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cf9e25c[Count = 0] remaining members to acquire global barrier 2023-02-09 17:55:51,455 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 17:55:51,455 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,456 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,456 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,456 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,457 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,457 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,44623,1675965244508' in zk 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35803,1675965244552' in zk 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 17:55:51,457 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43923,1675965244585' in zk 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,466 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 17:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,43923,1675965244585' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,35803,1675965244552' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,466 DEBUG [member: 'jenkins-hbase4.apache.org,44623,1675965244508' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 17:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43923,1675965244585': 2023-02-09 17:55:51,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43923,1675965244585' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 17:55:51,469 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 17:55:51,469 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 17:55:51,469 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 17:55:51,469 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 17:55:51,469 INFO [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,471 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,471 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 17:55:51,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,472 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,472 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 17:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,474 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,474 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 17:55:51,474 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 17:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,477 DEBUG [(jenkins-hbase4.apache.org,43955,1675965243365)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 17:55:51,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 17:55:51,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 17:55:51,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,477 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:51,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 17:55:51,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:51,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 17:55:51,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 17:55:51,478 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 17:55:51,578 DEBUG [Listener at localhost/34581] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 17:55:51,578 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:51,578 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:51,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 17:55:51,586 DEBUG [Listener at localhost/34581] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,586 DEBUG [Listener at localhost/34581] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=9811a03b0c4db1253c79047c0b600442, daughterA=95c74e50560d1934f89f9b7fd2372272, daughterB=78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:51,587 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=9811a03b0c4db1253c79047c0b600442, daughterA=95c74e50560d1934f89f9b7fd2372272, daughterB=78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:51,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=9811a03b0c4db1253c79047c0b600442, daughterA=95c74e50560d1934f89f9b7fd2372272, daughterB=78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:51,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=9811a03b0c4db1253c79047c0b600442, daughterA=95c74e50560d1934f89f9b7fd2372272, daughterB=78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:51,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, UNASSIGN}] 2023-02-09 17:55:51,590 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, UNASSIGN 2023-02-09 17:55:51,590 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=9811a03b0c4db1253c79047c0b600442, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:51,590 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965351590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965351590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965351590"}]},"ts":"1675965351590"} 2023-02-09 17:55:51,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 9811a03b0c4db1253c79047c0b600442, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:51,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9811a03b0c4db1253c79047c0b600442, disabling compactions & flushes 2023-02-09 17:55:51,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. after waiting 0 ms 2023-02-09 17:55:51,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 9811a03b0c4db1253c79047c0b600442 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 17:55:51,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/4c0add1b57f7437aa4dc555f0ff551b8 2023-02-09 17:55:51,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/a8e739a105ad455dbb043078ac1258f5 2023-02-09 17:55:51,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf1/4c0add1b57f7437aa4dc555f0ff551b8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8 2023-02-09 17:55:51,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:51,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/.tmp/cf2/a8e739a105ad455dbb043078ac1258f5 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5 2023-02-09 17:55:51,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 17:55:51,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 28ms, sequenceid=82, compaction requested=true 2023-02-09 17:55:51,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 17:55:51,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:51,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. 2023-02-09 17:55:51,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9811a03b0c4db1253c79047c0b600442: Waiting for close lock at 1675965351743Running coprocessor pre-close hooks at 1675965351743Disabling compacts and flushes for region at 1675965351743Disabling writes for close at 1675965351743Obtaining lock to block concurrent updates at 1675965351743Preparing flush snapshotting stores in 9811a03b0c4db1253c79047c0b600442 at 1675965351743Finished memstore snapshotting testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675965351743Flushing stores of testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. at 1675965351743Flushing 9811a03b0c4db1253c79047c0b600442/cf1: creating writer at 1675965351744 (+1 ms)Flushing 9811a03b0c4db1253c79047c0b600442/cf1: appending metadata at 1675965351746 (+2 ms)Flushing 9811a03b0c4db1253c79047c0b600442/cf1: closing flushed file at 1675965351746Flushing 9811a03b0c4db1253c79047c0b600442/cf2: creating writer at 1675965351755 (+9 ms)Flushing 9811a03b0c4db1253c79047c0b600442/cf2: appending metadata at 1675965351757 (+2 ms)Flushing 9811a03b0c4db1253c79047c0b600442/cf2: closing flushed file at 1675965351757Flushing 9811a03b0c4db1253c79047c0b600442/cf1: reopening flushed file at 1675965351765 (+8 ms)Flushing 9811a03b0c4db1253c79047c0b600442/cf2: reopening flushed file at 1675965351768 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 9811a03b0c4db1253c79047c0b600442 in 28ms, sequenceid=82, compaction requested=true at 1675965351771 (+3 ms)Writing region close event to WAL at 1675965351773 (+2 ms)Running coprocessor post-close hooks at 1675965351776 (+3 ms)Closed at 1675965351776 2023-02-09 17:55:51,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,777 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=9811a03b0c4db1253c79047c0b600442, regionState=CLOSED 2023-02-09 17:55:51,778 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965351777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965351777"}]},"ts":"1675965351777"} 2023-02-09 17:55:51,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-09 17:55:51,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 9811a03b0c4db1253c79047c0b600442, server=jenkins-hbase4.apache.org,43923,1675965244585 in 187 msec 2023-02-09 17:55:51,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-09 17:55:51,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=9811a03b0c4db1253c79047c0b600442, UNASSIGN in 191 msec 2023-02-09 17:55:51,789 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=9811a03b0c4db1253c79047c0b600442, threads=14 2023-02-09 17:55:51,789 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,789 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,789 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,789 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,789 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,790 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,873 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,874 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,877 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,878 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,878 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,880 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,881 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,881 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,883 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,883 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,886 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,887 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:51,888 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:52,236 ERROR [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 38633d84-bd4c-44ba-82f9-206cacf2085d) service to localhost/127.0.0.1:38249 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:55:52,242 ERROR [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6) service to localhost/127.0.0.1:38249 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 17:55:52,660 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7 for region: 9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:52,660 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 9811a03b0c4db1253c79047c0b600442 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 17:55:52,688 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:55:52,689 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 17:55:52,691 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675965352690"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675965352690"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675965352690"}]},"ts":"1675965352690"} 2023-02-09 17:55:52,691 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965352690"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965352690"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965352690"}]},"ts":"1675965352690"} 2023-02-09 17:55:52,691 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965352690"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965352690"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965352690"}]},"ts":"1675965352690"} 2023-02-09 17:55:52,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, ASSIGN}] 2023-02-09 17:55:52,694 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, ASSIGN 2023-02-09 17:55:52,694 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:55:52,695 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, ASSIGN 2023-02-09 17:55:52,695 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43923,1675965244585; forceNewPlan=false, retain=false 2023-02-09 17:55:52,845 INFO [jenkins-hbase4:43955] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 17:55:52,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=95c74e50560d1934f89f9b7fd2372272, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:52,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965352846"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965352846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965352846"}]},"ts":"1675965352846"} 2023-02-09 17:55:52,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 95c74e50560d1934f89f9b7fd2372272, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:52,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=78fcb7856a84317bd53c48ab8e057a15, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:52,850 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965352850"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965352850"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965352850"}]},"ts":"1675965352850"} 2023-02-09 17:55:52,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 78fcb7856a84317bd53c48ab8e057a15, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:53,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 78fcb7856a84317bd53c48ab8e057a15, NAME => 'testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 17:55:53,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:53,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:53,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,003 INFO [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,003 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1 2023-02-09 17:55:53,004 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1 2023-02-09 17:55:53,004 INFO [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78fcb7856a84317bd53c48ab8e057a15 columnFamilyName cf1 2023-02-09 17:55:53,015 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf-top 2023-02-09 17:55:53,017 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27-top 2023-02-09 17:55:53,020 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8-top 2023-02-09 17:55:53,023 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253-top 2023-02-09 17:55:53,026 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428-top 2023-02-09 17:55:53,029 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed-top 2023-02-09 17:55:53,032 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372-top 2023-02-09 17:55:53,032 INFO [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(310): Store=78fcb7856a84317bd53c48ab8e057a15/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:53,032 INFO [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,033 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2 2023-02-09 17:55:53,033 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2 2023-02-09 17:55:53,033 INFO [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78fcb7856a84317bd53c48ab8e057a15 columnFamilyName cf2 2023-02-09 17:55:53,044 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb-top 2023-02-09 17:55:53,047 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2-top 2023-02-09 17:55:53,050 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7-top 2023-02-09 17:55:53,053 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5-top 2023-02-09 17:55:53,056 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226-top 2023-02-09 17:55:53,059 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4-top 2023-02-09 17:55:53,064 DEBUG [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb-top 2023-02-09 17:55:53,065 INFO [StoreOpener-78fcb7856a84317bd53c48ab8e057a15-1] regionserver.HStore(310): Store=78fcb7856a84317bd53c48ab8e057a15/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:53,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:53,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 78fcb7856a84317bd53c48ab8e057a15; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63456406, jitterRate=-0.05442586541175842}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:53,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 78fcb7856a84317bd53c48ab8e057a15: Running coprocessor pre-open hook at 1675965353002Writing region info on filesystem at 1675965353002Initializing all the Stores at 1675965353002Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965353003 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965353003Cleaning up temporary data from old regions at 1675965353066 (+63 ms)Cleaning up detritus from prior splits at 1675965353066Running coprocessor post-open hooks at 1675965353068 (+2 ms)Region opened successfully at 1675965353068 2023-02-09 17:55:53,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15., pid=150, masterSystemTime=1675965352999 2023-02-09 17:55:53,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:53,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:53,070 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:53,070 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:53,070 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,070 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1912): 78fcb7856a84317bd53c48ab8e057a15/cf1 is initiating minor compaction (all files) 2023-02-09 17:55:53,070 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,070 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 78fcb7856a84317bd53c48ab8e057a15/cf1 in testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,070 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1912): 78fcb7856a84317bd53c48ab8e057a15/cf2 is initiating minor compaction (all files) 2023-02-09 17:55:53,070 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 78fcb7856a84317bd53c48ab8e057a15/cf2 in testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,070 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8-top] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/.tmp, totalSize=34.9 K 2023-02-09 17:55:53,070 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5-top] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/.tmp, totalSize=34.9 K 2023-02-09 17:55:53,071 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965350547 2023-02-09 17:55:53,071 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675965350547 2023-02-09 17:55:53,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,071 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965350725 2023-02-09 17:55:53,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 95c74e50560d1934f89f9b7fd2372272, NAME => 'testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 17:55:53,071 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675965350725 2023-02-09 17:55:53,072 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=78fcb7856a84317bd53c48ab8e057a15, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:53,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 17:55:53,072 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965350889 2023-02-09 17:55:53,072 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965353071"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965353071"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965353071"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965353071"}]},"ts":"1675965353071"} 2023-02-09 17:55:53,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,072 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675965350889 2023-02-09 17:55:53,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 17:55:53,072 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965351067 2023-02-09 17:55:53,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,072 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675965351067 2023-02-09 17:55:53,073 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675965351239 2023-02-09 17:55:53,073 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675965351239 2023-02-09 17:55:53,073 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675965351408 2023-02-09 17:55:53,073 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675965351408 2023-02-09 17:55:53,073 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] compactions.Compactor(207): Compacting 4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965351580 2023-02-09 17:55:53,074 INFO [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,074 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675965351580 2023-02-09 17:55:53,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-09 17:55:53,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 78fcb7856a84317bd53c48ab8e057a15, server=jenkins-hbase4.apache.org,43923,1675965244585 in 222 msec 2023-02-09 17:55:53,076 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1 2023-02-09 17:55:53,076 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1 2023-02-09 17:55:53,076 INFO [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95c74e50560d1934f89f9b7fd2372272 columnFamilyName cf1 2023-02-09 17:55:53,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, ASSIGN in 382 msec 2023-02-09 17:55:53,085 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 78fcb7856a84317bd53c48ab8e057a15#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:53,088 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] throttle.PressureAwareThroughputController(145): 78fcb7856a84317bd53c48ab8e057a15#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:53,091 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf-bottom 2023-02-09 17:55:53,101 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27-bottom 2023-02-09 17:55:53,104 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8-bottom 2023-02-09 17:55:53,106 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/.tmp/cf2/15f6d4f808e946aa9419aac5ee524488 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/15f6d4f808e946aa9419aac5ee524488 2023-02-09 17:55:53,106 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253-bottom 2023-02-09 17:55:53,109 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428-bottom 2023-02-09 17:55:53,110 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 78fcb7856a84317bd53c48ab8e057a15/cf2 of 78fcb7856a84317bd53c48ab8e057a15 into 15f6d4f808e946aa9419aac5ee524488(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:53,110 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 78fcb7856a84317bd53c48ab8e057a15: 2023-02-09 17:55:53,110 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15., storeName=78fcb7856a84317bd53c48ab8e057a15/cf2, priority=9, startTime=1675965353070; duration=0sec 2023-02-09 17:55:53,110 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:53,112 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed-bottom 2023-02-09 17:55:53,117 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372-bottom 2023-02-09 17:55:53,117 INFO [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(310): Store=95c74e50560d1934f89f9b7fd2372272/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:53,117 INFO [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,118 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2 2023-02-09 17:55:53,118 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2 2023-02-09 17:55:53,118 INFO [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95c74e50560d1934f89f9b7fd2372272 columnFamilyName cf2 2023-02-09 17:55:53,127 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb-bottom 2023-02-09 17:55:53,129 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2-bottom 2023-02-09 17:55:53,132 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7-bottom 2023-02-09 17:55:53,134 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5-bottom 2023-02-09 17:55:53,136 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226-bottom 2023-02-09 17:55:53,139 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4-bottom 2023-02-09 17:55:53,141 DEBUG [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(539): loaded hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb-bottom 2023-02-09 17:55:53,141 INFO [StoreOpener-95c74e50560d1934f89f9b7fd2372272-1] regionserver.HStore(310): Store=95c74e50560d1934f89f9b7fd2372272/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 17:55:53,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 17:55:53,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 95c74e50560d1934f89f9b7fd2372272; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59086578, jitterRate=-0.11954137682914734}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 17:55:53,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 95c74e50560d1934f89f9b7fd2372272: Running coprocessor pre-open hook at 1675965353072Writing region info on filesystem at 1675965353072Initializing all the Stores at 1675965353074 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965353074Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675965353074Cleaning up temporary data from old regions at 1675965353143 (+69 ms)Cleaning up detritus from prior splits at 1675965353143Running coprocessor post-open hooks at 1675965353145 (+2 ms)Region opened successfully at 1675965353145 2023-02-09 17:55:53,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272., pid=149, masterSystemTime=1675965352999 2023-02-09 17:55:53,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:53,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:53,147 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:53,147 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,147 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1912): 95c74e50560d1934f89f9b7fd2372272/cf1 is initiating minor compaction (all files) 2023-02-09 17:55:53,147 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 95c74e50560d1934f89f9b7fd2372272/cf1 in testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,148 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/.tmp, totalSize=34.9 K 2023-02-09 17:55:53,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,148 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965350547 2023-02-09 17:55:53,148 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=95c74e50560d1934f89f9b7fd2372272, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:53,149 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965353148"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675965353148"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675965353148"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675965353148"}]},"ts":"1675965353148"} 2023-02-09 17:55:53,149 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965350725 2023-02-09 17:55:53,149 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965350889 2023-02-09 17:55:53,150 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965351067 2023-02-09 17:55:53,150 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965351239 2023-02-09 17:55:53,151 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965351408 2023-02-09 17:55:53,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-09 17:55:53,151 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965351580 2023-02-09 17:55:53,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 95c74e50560d1934f89f9b7fd2372272, server=jenkins-hbase4.apache.org,43923,1675965244585 in 303 msec 2023-02-09 17:55:53,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-09 17:55:53,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, ASSIGN in 458 msec 2023-02-09 17:55:53,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=9811a03b0c4db1253c79047c0b600442, daughterA=95c74e50560d1934f89f9b7fd2372272, daughterB=78fcb7856a84317bd53c48ab8e057a15 in 1.5670 sec 2023-02-09 17:55:53,159 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] throttle.PressureAwareThroughputController(145): 95c74e50560d1934f89f9b7fd2372272#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:53,196 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/.tmp/cf1/6a58f99c09d44dc4bd06654ad66334c5 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/6a58f99c09d44dc4bd06654ad66334c5 2023-02-09 17:55:53,199 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 95c74e50560d1934f89f9b7fd2372272/cf1 of 95c74e50560d1934f89f9b7fd2372272 into 6a58f99c09d44dc4bd06654ad66334c5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:53,199 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 95c74e50560d1934f89f9b7fd2372272: 2023-02-09 17:55:53,200 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272., storeName=95c74e50560d1934f89f9b7fd2372272/cf1, priority=9, startTime=1675965353146; duration=0sec 2023-02-09 17:55:53,200 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 17:55:53,200 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 17:55:53,200 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,200 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1912): 95c74e50560d1934f89f9b7fd2372272/cf2 is initiating minor compaction (all files) 2023-02-09 17:55:53,200 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 95c74e50560d1934f89f9b7fd2372272/cf2 in testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,200 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5-bottom] into tmpdir=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/.tmp, totalSize=34.9 K 2023-02-09 17:55:53,201 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675965350547 2023-02-09 17:55:53,201 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675965350725 2023-02-09 17:55:53,201 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675965350889 2023-02-09 17:55:53,201 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675965351067 2023-02-09 17:55:53,202 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675965351239 2023-02-09 17:55:53,202 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting 772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675965351408 2023-02-09 17:55:53,202 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] compactions.Compactor(207): Compacting a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675965351580 2023-02-09 17:55:53,209 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] throttle.PressureAwareThroughputController(145): 95c74e50560d1934f89f9b7fd2372272#cf2#compaction#145 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 17:55:53,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43923] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:40196 deadline: 1675965413338, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442. is not online on jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:53,446 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:55:53,452 INFO [Listener at localhost/34581] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 17:55:53,453 DEBUG [Listener at localhost/34581] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 17:55:53,470 WARN [Listener at localhost/34581] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 17:55:53,470 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,470 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,471 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,471 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,472 INFO [Listener at localhost/34581] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-09 17:55:53,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-09 17:55:53,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,474 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,474 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 17:55:53,475 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965353475"}]},"ts":"1675965353475"} 2023-02-09 17:55:53,476 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-09 17:55:53,477 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-09 17:55:53,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, UNASSIGN}] 2023-02-09 17:55:53,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, UNASSIGN 2023-02-09 17:55:53,479 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=78fcb7856a84317bd53c48ab8e057a15, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:53,479 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965353479"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965353479"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965353479"}]},"ts":"1675965353479"} 2023-02-09 17:55:53,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 78fcb7856a84317bd53c48ab8e057a15, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:53,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, UNASSIGN 2023-02-09 17:55:53,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=95c74e50560d1934f89f9b7fd2372272, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:53,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965353483"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675965353483"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675965353483"}]},"ts":"1675965353483"} 2023-02-09 17:55:53,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 95c74e50560d1934f89f9b7fd2372272, server=jenkins-hbase4.apache.org,43923,1675965244585}] 2023-02-09 17:55:53,504 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/.tmp/cf1/3c23472189624f3983b4d95b2c141c54 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/3c23472189624f3983b4d95b2c141c54 2023-02-09 17:55:53,508 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 78fcb7856a84317bd53c48ab8e057a15/cf1 of 78fcb7856a84317bd53c48ab8e057a15 into 3c23472189624f3983b4d95b2c141c54(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:53,508 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 78fcb7856a84317bd53c48ab8e057a15: 2023-02-09 17:55:53,508 INFO [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15., storeName=78fcb7856a84317bd53c48ab8e057a15/cf1, priority=9, startTime=1675965353069; duration=0sec 2023-02-09 17:55:53,508 DEBUG [RS:2;jenkins-hbase4:43923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:53,575 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,575 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 17:55:53,618 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/.tmp/cf2/8a266c15967840d99390fde42ea2af08 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/8a266c15967840d99390fde42ea2af08 2023-02-09 17:55:53,621 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 95c74e50560d1934f89f9b7fd2372272/cf2 of 95c74e50560d1934f89f9b7fd2372272 into 8a266c15967840d99390fde42ea2af08(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 17:55:53,621 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 95c74e50560d1934f89f9b7fd2372272: 2023-02-09 17:55:53,621 INFO [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272., storeName=95c74e50560d1934f89f9b7fd2372272/cf2, priority=9, startTime=1675965353146; duration=0sec 2023-02-09 17:55:53,621 DEBUG [RS:2;jenkins-hbase4:43923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 17:55:53,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 78fcb7856a84317bd53c48ab8e057a15, disabling compactions & flushes 2023-02-09 17:55:53,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. after waiting 0 ms 2023-02-09 17:55:53,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,633 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8-top] to archive 2023-02-09 17:55:53,633 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:53,634 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,635 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,636 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,637 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,638 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,639 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,640 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,641 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7-top, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5-top] to archive 2023-02-09 17:55:53,642 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:53,643 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,644 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,645 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,646 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,647 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,648 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,649 DEBUG [StoreCloser-testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:55:53,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:53,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15. 2023-02-09 17:55:53,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 78fcb7856a84317bd53c48ab8e057a15: Waiting for close lock at 1675965353632Running coprocessor pre-close hooks at 1675965353632Disabling compacts and flushes for region at 1675965353632Disabling writes for close at 1675965353632Writing region close event to WAL at 1675965353649 (+17 ms)Running coprocessor post-close hooks at 1675965353652 (+3 ms)Closed at 1675965353652 2023-02-09 17:55:53,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 95c74e50560d1934f89f9b7fd2372272, disabling compactions & flushes 2023-02-09 17:55:53,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. after waiting 0 ms 2023-02-09 17:55:53,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,654 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=78fcb7856a84317bd53c48ab8e057a15, regionState=CLOSED 2023-02-09 17:55:53,654 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965353653"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965353653"}]},"ts":"1675965353653"} 2023-02-09 17:55:53,654 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8-bottom] to archive 2023-02-09 17:55:53,655 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:53,656 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f7e6f76588a34d5297df48ad59282372.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-09 17:55:53,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 78fcb7856a84317bd53c48ab8e057a15, server=jenkins-hbase4.apache.org,43923,1675965244585 in 175 msec 2023-02-09 17:55:53,657 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/223bfea8f56c47dd994490e33f0ccdaf.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=78fcb7856a84317bd53c48ab8e057a15, UNASSIGN in 178 msec 2023-02-09 17:55:53,658 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/c5ad951842c14b85bedfc01bdf554428.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,659 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/f433b8e508094de5bba3cc95478368ed.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,660 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/bc6a2b04b32546c191f68ade9b454253.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,662 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4487f45ff20f4e94a59543427db1eb27.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,663 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/4c0add1b57f7437aa4dc555f0ff551b8.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,665 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7-bottom, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442->hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5-bottom] to archive 2023-02-09 17:55:53,665 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 17:55:53,667 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/2aaa963af97b496c95a23216cd0ecabb.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,668 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/b74a4a29d13c4af1908ace075eda8226.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,669 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/d4f195e6dd2e4007bb8ab225e3cebea4.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,670 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/3e5969927a8d4cda8a7161fc376aecd2.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,672 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/e74297e27be342c4b1cc9908a25c3cfb.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,673 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/772b386070354d12b4e3b93b2cb081e7.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,675 DEBUG [StoreCloser-testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/a8e739a105ad455dbb043078ac1258f5.9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 17:55:53,679 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:53,679 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272. 2023-02-09 17:55:53,679 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 95c74e50560d1934f89f9b7fd2372272: Waiting for close lock at 1675965353653Running coprocessor pre-close hooks at 1675965353653Disabling compacts and flushes for region at 1675965353653Disabling writes for close at 1675965353653Writing region close event to WAL at 1675965353676 (+23 ms)Running coprocessor post-close hooks at 1675965353679 (+3 ms)Closed at 1675965353679 2023-02-09 17:55:53,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=95c74e50560d1934f89f9b7fd2372272, regionState=CLOSED 2023-02-09 17:55:53,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675965353681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675965353681"}]},"ts":"1675965353681"} 2023-02-09 17:55:53,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-09 17:55:53,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 95c74e50560d1934f89f9b7fd2372272, server=jenkins-hbase4.apache.org,43923,1675965244585 in 199 msec 2023-02-09 17:55:53,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-09 17:55:53,685 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=95c74e50560d1934f89f9b7fd2372272, UNASSIGN in 205 msec 2023-02-09 17:55:53,685 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675965353685"}]},"ts":"1675965353685"} 2023-02-09 17:55:53,689 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-09 17:55:53,690 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-09 17:55:53,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 219 msec 2023-02-09 17:55:53,776 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,776 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 17:55:53,777 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-09 17:55:53,777 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,777 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,778 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-09 17:55:53,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,780 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,780 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,780 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,781 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 17:55:53,784 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,784 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,784 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,786 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/recovered.edits] 2023-02-09 17:55:53,786 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/recovered.edits] 2023-02-09 17:55:53,786 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2, FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/recovered.edits] 2023-02-09 17:55:53,788 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/6a58f99c09d44dc4bd06654ad66334c5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf1/6a58f99c09d44dc4bd06654ad66334c5 2023-02-09 17:55:53,788 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/3c23472189624f3983b4d95b2c141c54 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf1/3c23472189624f3983b4d95b2c141c54 2023-02-09 17:55:53,789 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/223bfea8f56c47dd994490e33f0ccdaf 2023-02-09 17:55:53,791 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4487f45ff20f4e94a59543427db1eb27 2023-02-09 17:55:53,791 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/15f6d4f808e946aa9419aac5ee524488 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/cf2/15f6d4f808e946aa9419aac5ee524488 2023-02-09 17:55:53,791 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/8a266c15967840d99390fde42ea2af08 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/cf2/8a266c15967840d99390fde42ea2af08 2023-02-09 17:55:53,792 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/4c0add1b57f7437aa4dc555f0ff551b8 2023-02-09 17:55:53,793 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/bc6a2b04b32546c191f68ade9b454253 2023-02-09 17:55:53,793 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272/recovered.edits/90.seqid 2023-02-09 17:55:53,793 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/recovered.edits/90.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15/recovered.edits/90.seqid 2023-02-09 17:55:53,794 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/78fcb7856a84317bd53c48ab8e057a15 2023-02-09 17:55:53,794 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/95c74e50560d1934f89f9b7fd2372272 2023-02-09 17:55:53,794 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/c5ad951842c14b85bedfc01bdf554428 2023-02-09 17:55:53,795 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f433b8e508094de5bba3cc95478368ed 2023-02-09 17:55:53,796 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf1/f7e6f76588a34d5297df48ad59282372 2023-02-09 17:55:53,798 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/2aaa963af97b496c95a23216cd0ecabb 2023-02-09 17:55:53,798 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/3e5969927a8d4cda8a7161fc376aecd2 2023-02-09 17:55:53,799 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/772b386070354d12b4e3b93b2cb081e7 2023-02-09 17:55:53,800 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/a8e739a105ad455dbb043078ac1258f5 2023-02-09 17:55:53,801 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/b74a4a29d13c4af1908ace075eda8226 2023-02-09 17:55:53,802 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/d4f195e6dd2e4007bb8ab225e3cebea4 2023-02-09 17:55:53,803 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/cf2/e74297e27be342c4b1cc9908a25c3cfb 2023-02-09 17:55:53,806 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/recovered.edits/85.seqid to hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/archive/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442/recovered.edits/85.seqid 2023-02-09 17:55:53,806 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/.tmp/data/default/testSplitTableRegion/9811a03b0c4db1253c79047c0b600442 2023-02-09 17:55:53,806 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 17:55:53,808 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,810 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-09 17:55:53,811 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-09 17:55:53,812 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,812 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-09 17:55:53,812 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965353812"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:53,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965353812"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:53,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675965353812"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:53,814 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 17:55:53,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9811a03b0c4db1253c79047c0b600442, NAME => 'testSplitTableRegion,,1675965350043.9811a03b0c4db1253c79047c0b600442.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 95c74e50560d1934f89f9b7fd2372272, NAME => 'testSplitTableRegion,,1675965351586.95c74e50560d1934f89f9b7fd2372272.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 78fcb7856a84317bd53c48ab8e057a15, NAME => 'testSplitTableRegion,41,1675965351586.78fcb7856a84317bd53c48ab8e057a15.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 17:55:53,814 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-09 17:55:53,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675965353814"}]},"ts":"9223372036854775807"} 2023-02-09 17:55:53,815 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-09 17:55:53,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 17:55:53,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 40 msec 2023-02-09 17:55:53,881 INFO [Listener at localhost/34581] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 17:55:53,881 INFO [Listener at localhost/34581] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 17:55:53,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43955] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 17:55:53,882 INFO [Listener at localhost/34581] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-09 17:55:53,896 INFO [Listener at localhost/34581] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375 (was 375), OpenFileDescriptor=839 (was 845), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 223), ProcessCount=168 (was 168), AvailableMemoryMB=2897 (was 2949) 2023-02-09 17:55:53,896 INFO [Listener at localhost/34581] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 17:55:53,896 INFO [Listener at localhost/34581] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 17:55:53,899 DEBUG [Listener at localhost/34581] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4b731b7c to 127.0.0.1:50368 2023-02-09 17:55:53,900 DEBUG [Listener at localhost/34581] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:53,900 DEBUG [Listener at localhost/34581] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 17:55:53,901 DEBUG [Listener at localhost/34581] util.JVMClusterUtil(257): Found active master hash=1019338664, stopped=false 2023-02-09 17:55:53,901 INFO [Listener at localhost/34581] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:55:53,902 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 17:55:53,902 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 17:55:53,903 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 17:55:53,903 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 17:55:53,903 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:55:53,903 INFO [Listener at localhost/34581] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 17:55:53,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:55:53,903 DEBUG [Listener at localhost/34581] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1573a4e5 to 127.0.0.1:50368 2023-02-09 17:55:53,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:55:53,903 DEBUG [Listener at localhost/34581] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:53,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:55:53,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 17:55:53,904 INFO [Listener at localhost/34581] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,44623,1675965244508' ***** 2023-02-09 17:55:53,904 INFO [Listener at localhost/34581] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 17:55:53,904 INFO [Listener at localhost/34581] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35803,1675965244552' ***** 2023-02-09 17:55:53,904 INFO [Listener at localhost/34581] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 17:55:53,904 INFO [Listener at localhost/34581] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43923,1675965244585' ***** 2023-02-09 17:55:53,904 INFO [Listener at localhost/34581] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 17:55:53,904 INFO [RS:0;jenkins-hbase4:44623] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 17:55:53,904 INFO [RS:1;jenkins-hbase4:35803] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 17:55:53,904 INFO [RS:2;jenkins-hbase4:43923] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 17:55:53,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 17:55:53,905 INFO [RS:1;jenkins-hbase4:35803] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 17:55:53,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 17:55:53,905 INFO [RS:2;jenkins-hbase4:43923] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 17:55:53,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 17:55:53,905 INFO [RS:2;jenkins-hbase4:43923] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 17:55:53,905 INFO [RS:0;jenkins-hbase4:44623] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 17:55:53,905 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:53,905 INFO [RS:0;jenkins-hbase4:44623] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 17:55:53,905 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:53,905 DEBUG [RS:0;jenkins-hbase4:44623] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2f64f6e7 to 127.0.0.1:50368 2023-02-09 17:55:53,905 DEBUG [RS:0;jenkins-hbase4:44623] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:53,905 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,44623,1675965244508; all regions closed. 2023-02-09 17:55:53,905 INFO [RS:1;jenkins-hbase4:35803] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 17:55:53,905 DEBUG [RS:2;jenkins-hbase4:43923] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12e7c35f to 127.0.0.1:50368 2023-02-09 17:55:53,906 DEBUG [RS:2;jenkins-hbase4:43923] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:53,906 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(3304): Received CLOSE for b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:55:53,906 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43923,1675965244585; all regions closed. 2023-02-09 17:55:53,906 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:53,906 DEBUG [RS:1;jenkins-hbase4:35803] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a21b651 to 127.0.0.1:50368 2023-02-09 17:55:53,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b77a8e95cd64fdcc1c76452d991420d2, disabling compactions & flushes 2023-02-09 17:55:53,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:55:53,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:55:53,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. after waiting 0 ms 2023-02-09 17:55:53,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:55:53,906 DEBUG [RS:1;jenkins-hbase4:35803] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:53,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b77a8e95cd64fdcc1c76452d991420d2 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-09 17:55:53,906 INFO [RS:1;jenkins-hbase4:35803] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 17:55:53,906 INFO [RS:1;jenkins-hbase4:35803] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 17:55:53,907 INFO [RS:1;jenkins-hbase4:35803] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 17:55:53,907 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 17:55:53,907 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-09 17:55:53,907 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740, b77a8e95cd64fdcc1c76452d991420d2=hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2.} 2023-02-09 17:55:53,907 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 17:55:53,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 17:55:53,907 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 17:55:53,907 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 17:55:53,907 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 17:55:53,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-09 17:55:53,908 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1505): Waiting on 1588230740, b77a8e95cd64fdcc1c76452d991420d2 2023-02-09 17:55:54,061 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 17:55:54,061 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 17:55:54,061 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 17:55:54,061 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 17:55:54,061 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 17:55:54,061 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 17:55:54,062 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 17:55:54,064 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 17:55:54,062 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 17:55:54,069 DEBUG [RS:0;jenkins-hbase4:44623] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs 2023-02-09 17:55:54,070 INFO [RS:0;jenkins-hbase4:44623] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C44623%2C1675965244508:(num 1675965246080) 2023-02-09 17:55:54,070 DEBUG [RS:2;jenkins-hbase4:43923] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs 2023-02-09 17:55:54,070 INFO [RS:2;jenkins-hbase4:43923] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43923%2C1675965244585:(num 1675965246080) 2023-02-09 17:55:54,070 DEBUG [RS:2;jenkins-hbase4:43923] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:54,070 DEBUG [RS:0;jenkins-hbase4:44623] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:54,070 INFO [RS:2;jenkins-hbase4:43923] regionserver.LeaseManager(133): Closed leases 2023-02-09 17:55:54,070 INFO [RS:0;jenkins-hbase4:44623] regionserver.LeaseManager(133): Closed leases 2023-02-09 17:55:54,072 INFO [RS:2;jenkins-hbase4:43923] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 17:55:54,072 INFO [RS:0;jenkins-hbase4:44623] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 17:55:54,072 INFO [RS:2;jenkins-hbase4:43923] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 17:55:54,072 INFO [RS:0;jenkins-hbase4:44623] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 17:55:54,072 INFO [RS:0;jenkins-hbase4:44623] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 17:55:54,072 INFO [RS:0;jenkins-hbase4:44623] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 17:55:54,072 INFO [RS:2;jenkins-hbase4:43923] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 17:55:54,073 INFO [RS:2;jenkins-hbase4:43923] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 17:55:54,072 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 17:55:54,072 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 17:55:54,074 INFO [RS:2;jenkins-hbase4:43923] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43923 2023-02-09 17:55:54,074 INFO [RS:0;jenkins-hbase4:44623] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44623 2023-02-09 17:55:54,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/.tmp/info/25c730e378304b95ad8e4cff8571a8ba 2023-02-09 17:55:54,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 17:55:54,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:54,080 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:54,081 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e6357f7 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,081 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,44623,1675965244508] 2023-02-09 17:55:54,081 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 17:55:54,081 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a933d91 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,081 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:54,081 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@125c2d39 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,081 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@108e23ad rejected from java.util.concurrent.ThreadPoolExecutor@14b1b564[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,081 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,44623,1675965244508; numProcessing=1 2023-02-09 17:55:54,081 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43923,1675965244585 2023-02-09 17:55:54,081 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 17:55:54,082 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 17:55:54,082 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:54,083 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,44623,1675965244508 already deleted, retry=false 2023-02-09 17:55:54,084 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,44623,1675965244508 expired; onlineServers=2 2023-02-09 17:55:54,084 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43923,1675965244585] 2023-02-09 17:55:54,084 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43923,1675965244585; numProcessing=2 2023-02-09 17:55:54,085 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/.tmp/info/a847d14912c2487687404e7fc31810b8 2023-02-09 17:55:54,085 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43923,1675965244585 already deleted, retry=false 2023-02-09 17:55:54,085 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43923,1675965244585 expired; onlineServers=1 2023-02-09 17:55:54,086 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/.tmp/info/25c730e378304b95ad8e4cff8571a8ba as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/info/25c730e378304b95ad8e4cff8571a8ba 2023-02-09 17:55:54,082 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@15af4ca rejected from java.util.concurrent.ThreadPoolExecutor@14b1b564[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,088 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a847d14912c2487687404e7fc31810b8 2023-02-09 17:55:54,088 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,44623,1675965244508 2023-02-09 17:55:54,088 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@792bbff3 rejected from java.util.concurrent.ThreadPoolExecutor@14b1b564[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/info/25c730e378304b95ad8e4cff8571a8ba, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 17:55:54,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b77a8e95cd64fdcc1c76452d991420d2 in 184ms, sequenceid=6, compaction requested=false 2023-02-09 17:55:54,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/namespace/b77a8e95cd64fdcc1c76452d991420d2/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-09 17:55:54,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:54,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:55:54,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b77a8e95cd64fdcc1c76452d991420d2: Waiting for close lock at 1675965353906Running coprocessor pre-close hooks at 1675965353906Disabling compacts and flushes for region at 1675965353906Disabling writes for close at 1675965353906Obtaining lock to block concurrent updates at 1675965353906Preparing flush snapshotting stores in b77a8e95cd64fdcc1c76452d991420d2 at 1675965353906Finished memstore snapshotting hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675965353907 (+1 ms)Flushing stores of hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. at 1675965353912 (+5 ms)Flushing b77a8e95cd64fdcc1c76452d991420d2/info: creating writer at 1675965353912Flushing b77a8e95cd64fdcc1c76452d991420d2/info: appending metadata at 1675965354067 (+155 ms)Flushing b77a8e95cd64fdcc1c76452d991420d2/info: closing flushed file at 1675965354067Flushing b77a8e95cd64fdcc1c76452d991420d2/info: reopening flushed file at 1675965354086 (+19 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b77a8e95cd64fdcc1c76452d991420d2 in 184ms, sequenceid=6, compaction requested=false at 1675965354090 (+4 ms)Writing region close event to WAL at 1675965354091 (+1 ms)Running coprocessor post-close hooks at 1675965354097 (+6 ms)Closed at 1675965354097 2023-02-09 17:55:54,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675965246514.b77a8e95cd64fdcc1c76452d991420d2. 2023-02-09 17:55:54,100 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/.tmp/rep_barrier/62827557c70f452f8b4238e9889870d8 2023-02-09 17:55:54,103 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62827557c70f452f8b4238e9889870d8 2023-02-09 17:55:54,108 DEBUG [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 17:55:54,114 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/.tmp/table/63e4e596f7774f568577111ed19bee04 2023-02-09 17:55:54,116 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63e4e596f7774f568577111ed19bee04 2023-02-09 17:55:54,117 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/.tmp/info/a847d14912c2487687404e7fc31810b8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/info/a847d14912c2487687404e7fc31810b8 2023-02-09 17:55:54,120 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a847d14912c2487687404e7fc31810b8 2023-02-09 17:55:54,120 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/info/a847d14912c2487687404e7fc31810b8, entries=59, sequenceid=210, filesize=11.1 K 2023-02-09 17:55:54,120 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/.tmp/rep_barrier/62827557c70f452f8b4238e9889870d8 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/rep_barrier/62827557c70f452f8b4238e9889870d8 2023-02-09 17:55:54,123 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62827557c70f452f8b4238e9889870d8 2023-02-09 17:55:54,123 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/rep_barrier/62827557c70f452f8b4238e9889870d8, entries=24, sequenceid=210, filesize=7.5 K 2023-02-09 17:55:54,123 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/.tmp/table/63e4e596f7774f568577111ed19bee04 as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/table/63e4e596f7774f568577111ed19bee04 2023-02-09 17:55:54,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63e4e596f7774f568577111ed19bee04 2023-02-09 17:55:54,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/table/63e4e596f7774f568577111ed19bee04, entries=36, sequenceid=210, filesize=8.0 K 2023-02-09 17:55:54,127 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 220ms, sequenceid=210, compaction requested=false 2023-02-09 17:55:54,130 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-09 17:55:54,131 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 17:55:54,131 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 17:55:54,131 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 17:55:54,132 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675965353907Running coprocessor pre-close hooks at 1675965353907Disabling compacts and flushes for region at 1675965353907Disabling writes for close at 1675965353907Obtaining lock to block concurrent updates at 1675965353907Preparing flush snapshotting stores in 1588230740 at 1675965353907Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1675965353907Flushing stores of hbase:meta,,1.1588230740 at 1675965353912 (+5 ms)Flushing 1588230740/info: creating writer at 1675965353912Flushing 1588230740/info: appending metadata at 1675965354076 (+164 ms)Flushing 1588230740/info: closing flushed file at 1675965354077 (+1 ms)Flushing 1588230740/rep_barrier: creating writer at 1675965354088 (+11 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675965354091 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675965354091Flushing 1588230740/table: creating writer at 1675965354104 (+13 ms)Flushing 1588230740/table: appending metadata at 1675965354106 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675965354106Flushing 1588230740/info: reopening flushed file at 1675965354117 (+11 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675965354121 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675965354124 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 220ms, sequenceid=210, compaction requested=false at 1675965354127 (+3 ms)Writing region close event to WAL at 1675965354128 (+1 ms)Running coprocessor post-close hooks at 1675965354131 (+3 ms)Closed at 1675965354131 2023-02-09 17:55:54,132 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 17:55:54,180 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 17:55:54,182 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,182 INFO [RS:0;jenkins-hbase4:44623] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,44623,1675965244508; zookeeper connection closed. 2023-02-09 17:55:54,182 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@67368884 rejected from java.util.concurrent.ThreadPoolExecutor@14b1b564[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,183 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:44623-0x1003a3806680001, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,183 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@622fb36e rejected from java.util.concurrent.ThreadPoolExecutor@14b1b564[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,183 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@478343ad] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@478343ad 2023-02-09 17:55:54,282 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,282 INFO [RS:2;jenkins-hbase4:43923] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43923,1675965244585; zookeeper connection closed. 2023-02-09 17:55:54,282 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7c7d7949 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,283 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:43923-0x1003a3806680003, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,283 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@a0fa587] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@a0fa587 2023-02-09 17:55:54,283 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@9588759 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,308 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35803,1675965244552; all regions closed. 2023-02-09 17:55:54,312 DEBUG [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs 2023-02-09 17:55:54,312 INFO [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35803%2C1675965244552.meta:.meta(num 1675965246221) 2023-02-09 17:55:54,316 DEBUG [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/oldWALs 2023-02-09 17:55:54,317 INFO [RS:1;jenkins-hbase4:35803] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35803%2C1675965244552:(num 1675965246080) 2023-02-09 17:55:54,317 DEBUG [RS:1;jenkins-hbase4:35803] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:54,317 INFO [RS:1;jenkins-hbase4:35803] regionserver.LeaseManager(133): Closed leases 2023-02-09 17:55:54,317 INFO [RS:1;jenkins-hbase4:35803] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 17:55:54,317 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 17:55:54,317 INFO [RS:1;jenkins-hbase4:35803] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35803 2023-02-09 17:55:54,319 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35803,1675965244552 2023-02-09 17:55:54,319 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 17:55:54,319 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7886efe9 rejected from java.util.concurrent.ThreadPoolExecutor@7cb97421[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,320 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35803,1675965244552] 2023-02-09 17:55:54,320 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35803,1675965244552; numProcessing=3 2023-02-09 17:55:54,322 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35803,1675965244552 already deleted, retry=false 2023-02-09 17:55:54,322 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35803,1675965244552 expired; onlineServers=0 2023-02-09 17:55:54,322 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43955,1675965243365' ***** 2023-02-09 17:55:54,322 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 17:55:54,323 DEBUG [M:0;jenkins-hbase4:43955] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7dfcd593, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 17:55:54,323 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43955,1675965243365 2023-02-09 17:55:54,323 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43955,1675965243365; all regions closed. 2023-02-09 17:55:54,323 DEBUG [M:0;jenkins-hbase4:43955] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 17:55:54,323 DEBUG [M:0;jenkins-hbase4:43955] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 17:55:54,323 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 17:55:54,323 DEBUG [M:0;jenkins-hbase4:43955] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 17:55:54,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675965245689] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675965245689,5,FailOnTimeoutGroup] 2023-02-09 17:55:54,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675965245690] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675965245690,5,FailOnTimeoutGroup] 2023-02-09 17:55:54,323 INFO [M:0;jenkins-hbase4:43955] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 17:55:54,323 INFO [M:0;jenkins-hbase4:43955] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 17:55:54,323 INFO [M:0;jenkins-hbase4:43955] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 17:55:54,324 DEBUG [M:0;jenkins-hbase4:43955] master.HMaster(1502): Stopping service threads 2023-02-09 17:55:54,324 INFO [M:0;jenkins-hbase4:43955] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 17:55:54,324 ERROR [M:0;jenkins-hbase4:43955] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-09 17:55:54,324 INFO [M:0;jenkins-hbase4:43955] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 17:55:54,324 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 17:55:54,324 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 17:55:54,324 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 17:55:54,324 DEBUG [M:0;jenkins-hbase4:43955] zookeeper.ZKUtil(398): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 17:55:54,324 WARN [M:0;jenkins-hbase4:43955] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 17:55:54,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 17:55:54,324 INFO [M:0;jenkins-hbase4:43955] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 17:55:54,325 INFO [M:0;jenkins-hbase4:43955] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 17:55:54,325 DEBUG [M:0;jenkins-hbase4:43955] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 17:55:54,325 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:55:54,325 DEBUG [M:0;jenkins-hbase4:43955] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:55:54,325 DEBUG [M:0;jenkins-hbase4:43955] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 17:55:54,325 DEBUG [M:0;jenkins-hbase4:43955] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:55:54,325 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.92 KB heapSize=718.95 KB 2023-02-09 17:55:54,337 INFO [M:0;jenkins-hbase4:43955] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.92 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9b457e500f7f4bac8e1ce90316f2c8cd 2023-02-09 17:55:54,340 INFO [M:0;jenkins-hbase4:43955] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9b457e500f7f4bac8e1ce90316f2c8cd 2023-02-09 17:55:54,341 DEBUG [M:0;jenkins-hbase4:43955] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/9b457e500f7f4bac8e1ce90316f2c8cd as hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9b457e500f7f4bac8e1ce90316f2c8cd 2023-02-09 17:55:54,344 INFO [M:0;jenkins-hbase4:43955] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9b457e500f7f4bac8e1ce90316f2c8cd 2023-02-09 17:55:54,344 INFO [M:0;jenkins-hbase4:43955] regionserver.HStore(1080): Added hdfs://localhost:38249/user/jenkins/test-data/a12b1cf3-f7da-2adc-b49f-093f49b4fc07/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/9b457e500f7f4bac8e1ce90316f2c8cd, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-09 17:55:54,345 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegion(2947): Finished flush of dataSize ~607.92 KB/622515, heapSize ~718.93 KB/736184, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false 2023-02-09 17:55:54,346 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 17:55:54,346 DEBUG [M:0;jenkins-hbase4:43955] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675965354325Disabling compacts and flushes for region at 1675965354325Disabling writes for close at 1675965354325Obtaining lock to block concurrent updates at 1675965354325Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675965354325Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622515, getHeapSize=736184, getOffHeapSize=0, getCellsCount=1499 at 1675965354325Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675965354325Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675965354326 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675965354329 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675965354329Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675965354342 (+13 ms)Finished flush of dataSize ~607.92 KB/622515, heapSize ~718.93 KB/736184, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false at 1675965354345 (+3 ms)Writing region close event to WAL at 1675965354346 (+1 ms)Closed at 1675965354346 2023-02-09 17:55:54,349 INFO [M:0;jenkins-hbase4:43955] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 17:55:54,349 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 17:55:54,350 INFO [M:0;jenkins-hbase4:43955] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43955 2023-02-09 17:55:54,351 DEBUG [M:0;jenkins-hbase4:43955] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43955,1675965243365 already deleted, retry=false 2023-02-09 17:55:54,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,420 INFO [RS:1;jenkins-hbase4:35803] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35803,1675965244552; zookeeper connection closed. 2023-02-09 17:55:54,420 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@477ede23 rejected from java.util.concurrent.ThreadPoolExecutor@7cb97421[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,420 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): regionserver:35803-0x1003a3806680002, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,421 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@36108d92] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@36108d92 2023-02-09 17:55:54,421 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3f67fdd6 rejected from java.util.concurrent.ThreadPoolExecutor@7cb97421[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,421 INFO [Listener at localhost/34581] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-09 17:55:54,520 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,520 INFO [M:0;jenkins-hbase4:43955] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43955,1675965243365; zookeeper connection closed. 2023-02-09 17:55:54,520 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@743a5f1f rejected from java.util.concurrent.ThreadPoolExecutor@4a03580f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,521 DEBUG [Listener at localhost/34581-EventThread] zookeeper.ZKWatcher(600): master:43955-0x1003a3806680000, quorum=127.0.0.1:50368, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 17:55:54,521 ERROR [Listener at localhost/34581-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2770524f rejected from java.util.concurrent.ThreadPoolExecutor@4a03580f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 17:55:54,522 WARN [Listener at localhost/34581] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 17:55:54,525 INFO [Listener at localhost/34581] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 17:55:54,629 WARN [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 38633d84-bd4c-44ba-82f9-206cacf2085d) service to localhost/127.0.0.1:38249 2023-02-09 17:55:54,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data5/current/BP-393937471-172.31.14.131-1675965240225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 17:55:54,632 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data6/current/BP-393937471-172.31.14.131-1675965240225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 17:55:54,633 WARN [Listener at localhost/34581] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 17:55:54,635 INFO [Listener at localhost/34581] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 17:55:54,737 WARN [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 1270fd88-b6ef-45ff-b62a-d5cba4f7f7a6) service to localhost/127.0.0.1:38249 2023-02-09 17:55:54,738 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data3/current/BP-393937471-172.31.14.131-1675965240225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 17:55:54,738 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data4/current/BP-393937471-172.31.14.131-1675965240225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 17:55:54,739 WARN [Listener at localhost/34581] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 17:55:54,741 INFO [Listener at localhost/34581] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 17:55:54,849 WARN [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 17:55:54,849 WARN [BP-393937471-172.31.14.131-1675965240225 heartbeating to localhost/127.0.0.1:38249] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-393937471-172.31.14.131-1675965240225 (Datanode Uuid 634cd12e-a559-4a0e-a30b-33f9304f450f) service to localhost/127.0.0.1:38249 2023-02-09 17:55:54,850 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data1/current/BP-393937471-172.31.14.131-1675965240225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 17:55:54,850 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/48f0e675-6653-9a64-1a64-88cd74cf57fb/cluster_919fa01a-56ac-bece-3076-faf38f871175/dfs/data/data2/current/BP-393937471-172.31.14.131-1675965240225] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 17:55:54,872 INFO [Listener at localhost/34581] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 17:55:55,065 INFO [Listener at localhost/34581] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 17:55:55,301 INFO [Listener at localhost/34581] hbase.HBaseTestingUtility(1293): Minicluster is down